2024-04-20T20:59:45.373 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T20:59:45.373 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} --verbose --name rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396 -- /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/orig.config.yaml 2024-04-20T20:59:45.429 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T20:59:45.505 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396 branch: wip-rishabh-testing-20240416.193735 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7665396' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: a8c84f5997950fd7d008204839253291d52e6eda subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVLjm1H8OunaUtqlqULlwQaqtVnsxCBU0JXx1+QLcvtdZeRJdMPRlcz4Z/kKFcZKtKqyn+6db0D6wTRXtVVSUM= smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrt8hvUjTFHy0+87Ds8QyWCaE2LCQtXTyt/sewtNUTFCmziszohH3r/QhHUe5wvmRzLkl7CCdEkyyPD1hnLhk0= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIHplz7hWBXe4uK2eMcvNKEhfAiWfU5OH1plxMRpRIzKbzZ4iA+X9Fr86vGyaZ0Q11Stw89x+Xqw/OACmqEnwHI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_20:50:37 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T20:59:45.505 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T20:59:45.506 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T20:59:45.506 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T20:59:45.507 INFO:teuthology.task.internal:Checking packages... 2024-04-20T20:59:45.527 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T20:59:45.527 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T20:59:45.528 INFO:teuthology.packaging:ref: None 2024-04-20T20:59:45.528 INFO:teuthology.packaging:tag: None 2024-04-20T20:59:45.528 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T20:59:45.528 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T20:59:45.528 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T20:59:45.670 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T20:59:45.671 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T20:59:45.677 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T20:59:45.677 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T20:59:45.690 INFO:teuthology.task.internal:Saving configuration 2024-04-20T20:59:45.709 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T20:59:45.720 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T20:59:45.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.315764', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVLjm1H8OunaUtqlqULlwQaqtVnsxCBU0JXx1+QLcvtdZeRJdMPRlcz4Z/kKFcZKtKqyn+6db0D6wTRXtVVSUM='} 2024-04-20T20:59:45.760 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.317350', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrt8hvUjTFHy0+87Ds8QyWCaE2LCQtXTyt/sewtNUTFCmziszohH3r/QhHUe5wvmRzLkl7CCdEkyyPD1hnLhk0='} 2024-04-20T20:59:45.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.318624', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIHplz7hWBXe4uK2eMcvNKEhfAiWfU5OH1plxMRpRIzKbzZ4iA+X9Fr86vGyaZ0Q11Stw89x+Xqw/OACmqEnwHI='} 2024-04-20T20:59:45.783 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T20:59:45.794 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-20T20:59:45.794 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-20T20:59:45.794 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-20T20:59:45.794 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T20:59:45.895 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f60b4728160>, signals=[15]) 2024-04-20T20:59:45.895 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T20:59:45.903 INFO:teuthology.task.internal:Opening connections... 2024-04-20T20:59:45.904 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T20:59:45.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T20:59:45.987 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-04-20T20:59:45.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T20:59:46.064 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2024-04-20T20:59:46.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T20:59:46.139 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T20:59:46.148 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-04-20T20:59:46.167 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-04-20T20:59:46.167 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-04-20T20:59:46.224 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-04-20T20:59:46.224 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-04-20T20:59:46.224 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T20:59:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T20:59:46.226 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-04-20T20:59:46.251 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-04-20T20:59:46.270 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-04-20T20:59:46.270 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-04-20T20:59:46.328 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T20:59:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T20:59:46.329 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-20T20:59:46.351 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-04-20T20:59:46.368 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-04-20T20:59:46.368 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-04-20T20:59:46.426 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-04-20T20:59:46.426 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T20:59:46.427 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T20:59:46.428 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-04-20T20:59:46.451 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T20:59:46.465 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T20:59:46.479 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T20:59:46.479 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-04-20T20:59:46.481 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-04-20T20:59:46.484 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2024-04-20T20:59:46.501 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T20:59:46.513 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T20:59:46.513 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-04-20T20:59:46.539 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-04-20T20:59:46.542 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2024-04-20T20:59:46.573 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T20:59:46.644 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T20:59:46.660 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T20:59:46.660 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-20T20:59:46.660 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-20T20:59:46.661 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T20:59:46.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T20:59:46.678 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-20T20:59:46.952 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:59:46.952 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T20:59:46.952 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-04-20T20:59:46.970 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-437.el9.x86_64 2024-04-20T20:59:46.970 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:59:46.971 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T20:59:46.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2024-04-20T20:59:50.120 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-20T21:00:14.068 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 852 kB/s | 19 MB 00:22 2024-04-20T21:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-04-20T21:00:24.119 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-20T21:00:25.395 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-20T21:00:30.587 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-20T21:00:32.714 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-04-20T21:00:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-04-20T21:00:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-04-20T21:00:32.819 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:00:32.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T21:00:33.063 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T21:00:33.064 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T21:00:33.067 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2024-04-20T21:09:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20 21:09:27 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T21:09:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2024-04-20 21:09:27-- 2024-04-20T21:09:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 8m 55s 2024-04-20T21:09:27.979 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 341M in 8m 54s (653 KB/s) 2024-04-20T21:09:27.982 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-20T21:09:27.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T21:09:30.186 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2024-04-20T21:09:30.417 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2024-04-20T21:09:30.536 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2024-04-20T21:09:38.927 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T21:10:17.294 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T21:10:17.295 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T21:10:17.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T21:10:17.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T21:10:17.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T21:10:17.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T21:10:17.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T21:10:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T21:10:17.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T21:10:17.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T21:10:17.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T21:10:17.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T21:10:17.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T21:10:17.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T21:10:17.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T21:10:17.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T21:10:17.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T21:10:17.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T21:10:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T21:10:17.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T21:10:17.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T21:10:17.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T21:10:17.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T21:10:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T21:10:17.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T21:10:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T21:10:17.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T21:10:17.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T21:10:17.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T21:10:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T21:10:17.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T21:10:17.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T21:10:17.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T21:10:17.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T21:10:17.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T21:10:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T21:10:17.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T21:10:17.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T21:10:17.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T21:10:17.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T21:10:17.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T21:10:17.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T21:10:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T21:10:17.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T21:10:17.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T21:10:17.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T21:10:17.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T21:10:17.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T21:10:17.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T21:10:17.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T21:10:17.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T21:10:17.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T21:10:17.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T21:10:17.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T21:10:17.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T21:10:17.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T21:10:17.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T21:10:17.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T21:10:17.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T21:10:17.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T21:10:17.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T21:10:17.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T21:10:17.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T21:10:17.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T21:10:17.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T21:10:17.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T21:10:17.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T21:10:17.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T21:10:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T21:10:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T21:10:17.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T21:10:17.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T21:10:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T21:10:17.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T21:10:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T21:10:17.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T21:10:17.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T21:10:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T21:10:17.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T21:10:17.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T21:10:17.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T21:10:17.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T21:10:17.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T21:10:17.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T21:10:17.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T21:10:17.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T21:10:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T21:10:17.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T21:10:17.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T21:10:17.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T21:10:17.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T21:10:17.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T21:10:17.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T21:10:17.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T21:10:17.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T21:10:17.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T21:10:17.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T21:10:17.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T21:10:17.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T21:10:17.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T21:10:17.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T21:10:17.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T21:10:17.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T21:10:17.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T21:10:17.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T21:10:17.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T21:10:17.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T21:10:17.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T21:10:17.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T21:10:17.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T21:10:17.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T21:10:17.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T21:10:17.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T21:10:17.484 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T21:10:17.485 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T21:10:17.486 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T21:10:17.487 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T21:10:17.488 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T21:10:17.489 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T21:10:17.490 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T21:10:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T21:10:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T21:10:17.493 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T21:10:17.494 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T21:10:17.495 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T21:10:17.496 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T21:10:17.497 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T21:10:17.498 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T21:10:17.499 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T21:10:17.500 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T21:10:17.501 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T21:10:17.502 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T21:10:17.503 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T21:10:17.504 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T21:10:17.505 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T21:10:17.506 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T21:10:17.507 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T21:10:17.508 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T21:10:17.509 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T21:10:17.510 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:10:17.510 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:10:17.514 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.514 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T21:10:17.538 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T21:10:17.538 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:10:17.538 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:10:17.538 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-20T21:10:17.538 INFO:teuthology.orchestra.run.smithi007.stdout:for file in vmlinuz System.map config; do 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: fi 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T21:10:17.539 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:10:17.540 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.540 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-20T21:10:17.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-04-20T21:10:17.622 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-04-20T21:10:17.622 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-04-20T21:10:17.622 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T21:10:17.623 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:10:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T21:10:17.625 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T21:10:17.625 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:17.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T21:10:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-04-20T21:10:19.597 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T21:10:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-04-20T21:10:19.630 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-04-20T21:10:19.645 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.0hpltnmqAR 2024-04-20T21:10:19.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0hpltnmqAR 2024-04-20T21:10:19.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.0hpltnmqAR 2024-04-20T21:10:19.859 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.0hpltnmqAR is 6KB 2024-04-20T21:10:19.908 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.0hpltnmqAR 2024-04-20T21:10:19.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T21:10:19.997 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T21:10:19.998 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T21:10:19.998 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T21:10:19.998 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T21:10:19.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:20.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-04-20T21:10:50.164 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-20T21:10:50.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:10:50.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:11:08.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-20T21:11:17.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:11:17.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:11:20.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-20T21:11:32.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:11:32.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:11:51.074 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-20T21:12:06.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:12:06.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:12:06.526 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-04-20T21:12:07.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:12:07.183 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T21:12:07.183 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-04-20T21:12:07.226 INFO:teuthology.orchestra.run.smithi007.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T21:12:07.226 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T21:12:07.226 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T21:12:07.226 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T21:12:08.227 DEBUG:teuthology.parallel:result is None 2024-04-20T21:12:08.228 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T21:12:08.235 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T21:12:08.236 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:12:08.241 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:12:08.243 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:12:08.280 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T21:12:08.287 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T21:12:08.294 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T21:12:08.294 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:12:08.334 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:12:08.339 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:12:08.459 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T21:12:08.467 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T21:12:08.467 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:12:08.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:12:08.486 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T21:12:08.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:12:08.503 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:12:08.509 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:12:08.528 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:12:08.565 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.579 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.609 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.630 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.682 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.770 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:12:08.775 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T21:12:08.782 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T21:12:08.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:12:08.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:12:08.789 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:12:08.860 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T21:12:08.869 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T21:12:08.870 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:12:08.874 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:12:08.877 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:12:08.916 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.134 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.299 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:12:09.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:12:09.388 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.481 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.610 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:12:09.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:12:09.677 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.721 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:12:09.783 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.827 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:12:09.868 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:12:09.869 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:12:09.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-04-20T21:12:09.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-04-20T21:12:09.943 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2024-04-20T21:12:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:12:10.010 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:12:10.480 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T21:12:10.492 INFO:teuthology.task.internal:Starting timer... 2024-04-20T21:12:10.492 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T21:12:10.589 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T21:12:10.598 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-20T21:12:10.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-04-20T21:12:10.723 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-04-20T21:12:10.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-04-20T21:12:10.809 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-04-20T21:12:10.812 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2024-04-20T21:12:10.878 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2024-04-20T21:12:10.880 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T21:12:10.880 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-04-20T21:12:10.927 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-04-20T21:12:10.928 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-04-20T21:12:10.958 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-04-20T21:12:10.958 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2024-04-20T21:12:10.981 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2024-04-20T21:12:10.981 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2024-04-20T21:12:10.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:12:11.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:12:11.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:12:11.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:12:11.078 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:12:11.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:12:11.107 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T21:12:11.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:12:11.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:12:11.201 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:12:11.225 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T21:12:11.236 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T21:12:11.253 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T21:12:11.254 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi016.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T21:14:41.376 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2024-04-20T21:14:41.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:14:41.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:41.502 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-04-20T21:14:41.667 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-20T21:14:41.668 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-20T21:14:41.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:41.749 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-20T21:14:41.836 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-20T21:14:41.837 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-20T21:14:41.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:14:41.918 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-04-20T21:14:41.955 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-04-20T21:14:41.956 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T21:14:41.968 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T21:14:41.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:41.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:41.977 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:41.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:41.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:14:41.979 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:14:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:42.023 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:42.036 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:42.051 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-04-20T21:14:42.058 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2024-04-20T21:14:42.062 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-04-20T21:14:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:14:42.068 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2024-04-20T21:14:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:42.082 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:42.091 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:42.095 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:14:42.153 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:42.157 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:42.157 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-20T21:14:42.157 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.158 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.158 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.158 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:42.161 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:42.161 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-20T21:14:42.162 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.162 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.162 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.162 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-04-20T21:14:42.225 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-04-20T21:14:42.269 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:14:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:14:42.450 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:14:42.459 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:14:42.459 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-04-20T21:14:42.460 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.460 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.460 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.460 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:14:42.461 INFO:teuthology.run_tasks:Running task install... 2024-04-20T21:14:42.470 DEBUG:teuthology.task.install:project ceph 2024-04-20T21:14:42.470 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T21:14:42.470 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T21:14:42.470 INFO:teuthology.task.install:Using flavor: default 2024-04-20T21:14:42.477 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T21:14:42.477 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T21:14:42.478 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:42.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:42.480 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:42.480 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:42.481 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:14:42.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:14:42.628 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:42.628 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:42.634 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:42.634 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:42.697 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:14:42.697 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:14:42.943 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:42.943 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:14:42.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:42.972 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:42.972 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:14:42.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:43.002 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:43.002 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:43.007 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:14:43.007 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:14:43.007 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:14:43.038 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:43.038 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:43.039 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:14:43.039 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:14:43.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:43.108 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:43.150 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:43.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:14:43.184 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:43.215 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-04-20T21:14:43.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-04-20T21:14:43.248 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2024-04-20T21:14:43.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2024-04-20T21:14:43.298 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:14:43.385 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-04-20T21:14:43.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-04-20T21:14:43.454 INFO:teuthology.orchestra.run.smithi016.stdout:48 files removed 2024-04-20T21:14:43.479 INFO:teuthology.orchestra.run.smithi186.stdout:48 files removed 2024-04-20T21:14:43.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:43.507 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:43.747 INFO:teuthology.orchestra.run.smithi007.stdout:48 files removed 2024-04-20T21:14:43.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:14:44.621 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 98 kB/s | 84 kB 00:00 2024-04-20T21:14:44.624 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 96 kB/s | 84 kB 00:00 2024-04-20T21:14:45.172 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 89 kB/s | 84 kB 00:00 2024-04-20T21:14:45.404 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-20T21:14:45.483 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-04-20T21:14:46.055 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-04-20T21:14:46.147 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-20T21:14:46.267 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-20T21:14:47.031 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-04-20T21:14:47.130 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-20T21:14:48.296 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-20T21:14:49.015 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-20T21:14:49.919 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-04-20T21:14:50.464 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-20T21:14:53.790 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.2 MB 00:01 2024-04-20T21:14:56.289 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-04-20T21:14:56.290 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-20T21:14:57.659 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-20T21:14:58.066 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-20T21:14:59.558 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-20T21:15:02.709 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-20T21:15:04.884 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.885 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.896 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:15:04.897 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:15:04.897 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:15:04.898 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:15:04.900 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:15:04.902 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:15:04.903 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:15:04.962 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2024-04-20T21:15:04.979 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-20T21:15:04.979 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:15:04.980 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:15:04.981 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:15:04.982 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:15:04.983 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:15:04.984 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:15:04.985 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:15:04.986 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:15:04.987 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:15:04.988 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:15:04.989 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:15:04.990 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout:Installing weak dependencies: 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout:======================================================================================= 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout:Install 148 Packages 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 2 Packages 2024-04-20T21:15:04.991 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:15:04.992 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 218 M 2024-04-20T21:15:04.992 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2024-04-20T21:15:05.769 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-20T21:15:06.160 INFO:teuthology.orchestra.run.smithi186.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-20T21:15:07.918 INFO:teuthology.orchestra.run.smithi186.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 508 kB/s | 893 kB 00:01 2024-04-20T21:15:08.160 INFO:teuthology.orchestra.run.smithi186.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 600 kB/s | 145 kB 00:00 2024-04-20T21:15:08.287 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:08.288 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:08.309 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:15:08.310 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:15:08.311 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:15:08.313 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:15:08.315 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:15:08.319 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:15:08.320 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:15:08.395 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-04-20T21:15:08.410 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:15:08.411 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:15:08.412 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:15:08.413 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:15:08.414 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:15:08.415 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:15:08.416 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:15:08.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:15:08.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:15:08.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:15:08.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:15:08.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:15:08.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:15:08.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:15:08.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:15:08.425 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-20T21:15:08.426 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-04-20T21:15:08.427 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-04-20T21:15:08.427 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:15:08.428 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 218 M 2024-04-20T21:15:08.428 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-04-20T21:15:09.721 INFO:teuthology.orchestra.run.smithi007.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-20T21:15:11.468 INFO:teuthology.orchestra.run.smithi186.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 731 kB/s | 2.4 MB 00:03 2024-04-20T21:15:12.616 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 902 kB/s | 19 MB 00:21 2024-04-20T21:15:12.737 INFO:teuthology.orchestra.run.smithi007.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 296 kB/s | 893 kB 00:03 2024-04-20T21:15:13.062 INFO:teuthology.orchestra.run.smithi007.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 446 kB/s | 145 kB 00:00 2024-04-20T21:15:15.935 INFO:teuthology.orchestra.run.smithi186.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 343 kB/s | 1.5 MB 00:04 2024-04-20T21:15:16.719 INFO:teuthology.orchestra.run.smithi186.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 501 kB/s | 5.4 MB 00:10 2024-04-20T21:15:17.252 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 6.2 MB 00:00 2024-04-20T21:15:17.287 INFO:teuthology.orchestra.run.smithi007.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 572 kB/s | 2.4 MB 00:04 2024-04-20T21:15:18.750 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-20T21:15:19.979 INFO:teuthology.orchestra.run.smithi007.stdout:(5/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 514 kB/s | 5.4 MB 00:10 2024-04-20T21:15:20.087 INFO:teuthology.orchestra.run.smithi007.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 548 kB/s | 1.5 MB 00:02 2024-04-20T21:15:20.385 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-20T21:15:24.529 INFO:teuthology.orchestra.run.smithi186.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 560 kB/s | 4.7 MB 00:08 2024-04-20T21:15:24.829 INFO:teuthology.orchestra.run.smithi007.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 992 kB/s | 4.7 MB 00:04 2024-04-20T21:15:25.585 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-20T21:15:27.854 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:27.854 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:15:27.866 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:15:27.867 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:15:27.868 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:15:27.869 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:15:27.871 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:15:27.873 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:15:27.874 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:15:27.939 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-04-20T21:15:27.954 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-20T21:15:27.954 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-04-20T21:15:27.954 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-20T21:15:27.955 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-04-20T21:15:27.955 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:15:27.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:15:27.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:27.955 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:15:27.956 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:15:27.957 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:15:27.958 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-04-20T21:15:27.959 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:15:27.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:15:27.961 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:15:27.962 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:15:27.963 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:15:27.964 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:15:27.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:15:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:15:27.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:15:27.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:15:27.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:15:27.970 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-04-20T21:15:27.971 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-04-20T21:15:27.972 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-04-20T21:15:27.972 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2024-04-20T21:15:27.972 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:15:27.974 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 218 M 2024-04-20T21:15:27.974 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-04-20T21:15:29.136 INFO:teuthology.orchestra.run.smithi016.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-20T21:15:30.295 INFO:teuthology.orchestra.run.smithi016.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 771 kB/s | 893 kB 00:01 2024-04-20T21:15:30.470 INFO:teuthology.orchestra.run.smithi016.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 829 kB/s | 145 kB 00:00 2024-04-20T21:15:32.945 INFO:teuthology.orchestra.run.smithi016.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 977 kB/s | 2.4 MB 00:02 2024-04-20T21:15:34.604 INFO:teuthology.orchestra.run.smithi016.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 925 kB/s | 1.5 MB 00:01 2024-04-20T21:15:38.470 INFO:teuthology.orchestra.run.smithi016.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 566 kB/s | 5.4 MB 00:09 2024-04-20T21:15:40.184 INFO:teuthology.orchestra.run.smithi186.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 628 kB/s | 21 MB 00:34 2024-04-20T21:15:40.284 INFO:teuthology.orchestra.run.smithi186.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 253 kB/s | 25 kB 00:00 2024-04-20T21:15:41.036 INFO:teuthology.orchestra.run.smithi007.stdout:(8/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x86 823 kB/s | 17 MB 00:20 2024-04-20T21:15:41.528 INFO:teuthology.orchestra.run.smithi016.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 695 kB/s | 4.7 MB 00:06 2024-04-20T21:15:41.561 INFO:teuthology.orchestra.run.smithi007.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 48 kB/s | 25 kB 00:00 2024-04-20T21:15:42.367 INFO:teuthology.orchestra.run.smithi186.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 608 kB/s | 11 MB 00:17 2024-04-20T21:15:42.768 INFO:teuthology.orchestra.run.smithi186.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 562 kB/s | 225 kB 00:00 2024-04-20T21:15:42.876 INFO:teuthology.orchestra.run.smithi186.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 249 kB/s | 27 kB 00:00 2024-04-20T21:15:42.984 INFO:teuthology.orchestra.run.smithi186.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 293 kB/s | 32 kB 00:00 2024-04-20T21:15:44.334 INFO:teuthology.orchestra.run.smithi186.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 554 kB/s | 748 kB 00:01 2024-04-20T21:15:44.676 INFO:teuthology.orchestra.run.smithi186.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 478 kB/s | 163 kB 00:00 2024-04-20T21:15:44.935 INFO:teuthology.orchestra.run.smithi186.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 490 kB/s | 126 kB 00:00 2024-04-20T21:15:45.818 INFO:teuthology.orchestra.run.smithi186.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 567 kB/s | 501 kB 00:00 2024-04-20T21:15:48.852 INFO:teuthology.orchestra.run.smithi186.stdout:(18/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 536 kB/s | 17 MB 00:32 2024-04-20T21:15:49.027 INFO:teuthology.orchestra.run.smithi186.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 258 kB/s | 45 kB 00:00 2024-04-20T21:15:49.369 INFO:teuthology.orchestra.run.smithi186.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 394 kB/s | 135 kB 00:00 2024-04-20T21:15:49.535 INFO:teuthology.orchestra.run.smithi007.stdout:(10/150): ceph-common-19.0.0-3133.ga8c84f59.el9 538 kB/s | 21 MB 00:40 2024-04-20T21:15:49.760 INFO:teuthology.orchestra.run.smithi186.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 414 kB/s | 162 kB 00:00 2024-04-20T21:15:50.385 INFO:teuthology.orchestra.run.smithi007.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 265 kB/s | 225 kB 00:00 2024-04-20T21:15:50.411 INFO:teuthology.orchestra.run.smithi186.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 494 kB/s | 321 kB 00:00 2024-04-20T21:15:50.535 INFO:teuthology.orchestra.run.smithi007.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 181 kB/s | 27 kB 00:00 2024-04-20T21:15:50.986 INFO:teuthology.orchestra.run.smithi186.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 519 kB/s | 299 kB 00:00 2024-04-20T21:15:51.319 INFO:teuthology.orchestra.run.smithi007.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 41 kB/s | 32 kB 00:00 2024-04-20T21:15:51.386 INFO:teuthology.orchestra.run.smithi186.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 249 kB/s | 99 kB 00:00 2024-04-20T21:15:51.878 INFO:teuthology.orchestra.run.smithi186.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 173 kB/s | 85 kB 00:00 2024-04-20T21:15:53.411 INFO:teuthology.orchestra.run.smithi007.stdout:(14/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 380 kB/s | 11 MB 00:28 2024-04-20T21:15:53.894 INFO:teuthology.orchestra.run.smithi007.stdout:(15/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 291 kB/s | 748 kB 00:02 2024-04-20T21:15:54.294 INFO:teuthology.orchestra.run.smithi007.stdout:(16/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 185 kB/s | 163 kB 00:00 2024-04-20T21:15:54.411 INFO:teuthology.orchestra.run.smithi007.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 245 kB/s | 126 kB 00:00 2024-04-20T21:15:56.253 INFO:teuthology.orchestra.run.smithi007.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 256 kB/s | 501 kB 00:01 2024-04-20T21:15:56.628 INFO:teuthology.orchestra.run.smithi186.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 509 kB/s | 5.4 MB 00:10 2024-04-20T21:15:56.870 INFO:teuthology.orchestra.run.smithi007.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 73 kB/s | 45 kB 00:00 2024-04-20T21:15:57.203 INFO:teuthology.orchestra.run.smithi186.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 296 kB/s | 170 kB 00:00 2024-04-20T21:15:57.378 INFO:teuthology.orchestra.run.smithi186.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 195 kB/s | 34 kB 00:00 2024-04-20T21:15:57.545 INFO:teuthology.orchestra.run.smithi007.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 200 kB/s | 135 kB 00:00 2024-04-20T21:15:57.778 INFO:teuthology.orchestra.run.smithi186.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 356 kB/s | 142 kB 00:00 2024-04-20T21:15:58.262 INFO:teuthology.orchestra.run.smithi007.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 226 kB/s | 162 kB 00:00 2024-04-20T21:15:58.595 INFO:teuthology.orchestra.run.smithi016.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 636 kB/s | 11 MB 00:17 2024-04-20T21:15:58.703 INFO:teuthology.orchestra.run.smithi016.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 233 kB/s | 25 kB 00:00 2024-04-20T21:15:59.470 INFO:teuthology.orchestra.run.smithi007.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 266 kB/s | 321 kB 00:01 2024-04-20T21:15:59.753 INFO:teuthology.orchestra.run.smithi186.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 402 kB/s | 3.1 MB 00:07 2024-04-20T21:16:00.254 INFO:teuthology.orchestra.run.smithi007.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 382 kB/s | 299 kB 00:00 2024-04-20T21:16:00.587 INFO:teuthology.orchestra.run.smithi007.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 299 kB/s | 99 kB 00:00 2024-04-20T21:16:00.821 INFO:teuthology.orchestra.run.smithi007.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 366 kB/s | 85 kB 00:00 2024-04-20T21:16:05.662 INFO:teuthology.orchestra.run.smithi186.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 460 kB/s | 3.5 MB 00:07 2024-04-20T21:16:06.153 INFO:teuthology.orchestra.run.smithi186.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 506 kB/s | 249 kB 00:00 2024-04-20T21:16:06.337 INFO:teuthology.orchestra.run.smithi186.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 269 kB/s | 49 kB 00:00 2024-04-20T21:16:06.437 INFO:teuthology.orchestra.run.smithi186.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 165 kB/s | 16 kB 00:00 2024-04-20T21:16:07.096 INFO:teuthology.orchestra.run.smithi186.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 415 kB/s | 273 kB 00:00 2024-04-20T21:16:08.429 INFO:teuthology.orchestra.run.smithi186.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 572 kB/s | 763 kB 00:01 2024-04-20T21:16:08.530 INFO:teuthology.orchestra.run.smithi186.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 182 kB/s | 18 kB 00:00 2024-04-20T21:16:08.680 INFO:teuthology.orchestra.run.smithi186.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 538 kB/s | 80 kB 00:00 2024-04-20T21:16:08.838 INFO:teuthology.orchestra.run.smithi186.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.0 MB/s | 803 kB 00:00 2024-04-20T21:16:08.897 INFO:teuthology.orchestra.run.smithi186.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-20T21:16:08.956 INFO:teuthology.orchestra.run.smithi186.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-20T21:16:09.056 INFO:teuthology.orchestra.run.smithi186.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-20T21:16:09.114 INFO:teuthology.orchestra.run.smithi186.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T21:16:09.173 INFO:teuthology.orchestra.run.smithi186.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-20T21:16:09.257 INFO:teuthology.orchestra.run.smithi186.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-20T21:16:09.315 INFO:teuthology.orchestra.run.smithi186.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-20T21:16:09.524 INFO:teuthology.orchestra.run.smithi186.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-20T21:16:09.599 INFO:teuthology.orchestra.run.smithi186.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-20T21:16:09.649 INFO:teuthology.orchestra.run.smithi186.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-20T21:16:09.700 INFO:teuthology.orchestra.run.smithi186.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-20T21:16:09.750 INFO:teuthology.orchestra.run.smithi186.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-20T21:16:09.800 INFO:teuthology.orchestra.run.smithi186.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-20T21:16:09.917 INFO:teuthology.orchestra.run.smithi186.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-04-20T21:16:09.968 INFO:teuthology.orchestra.run.smithi186.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-20T21:16:10.018 INFO:teuthology.orchestra.run.smithi186.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-20T21:16:10.068 INFO:teuthology.orchestra.run.smithi186.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-20T21:16:10.118 INFO:teuthology.orchestra.run.smithi186.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-20T21:16:10.177 INFO:teuthology.orchestra.run.smithi186.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-20T21:16:10.227 INFO:teuthology.orchestra.run.smithi186.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-20T21:16:10.278 INFO:teuthology.orchestra.run.smithi186.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 510 kB/s | 25 kB 00:00 2024-04-20T21:16:10.336 INFO:teuthology.orchestra.run.smithi186.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-04-20T21:16:10.387 INFO:teuthology.orchestra.run.smithi186.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-20T21:16:10.445 INFO:teuthology.orchestra.run.smithi186.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-20T21:16:10.495 INFO:teuthology.orchestra.run.smithi186.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-20T21:16:10.654 INFO:teuthology.orchestra.run.smithi186.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-20T21:16:10.788 INFO:teuthology.orchestra.run.smithi186.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 31 MB/s | 4.1 MB 00:00 2024-04-20T21:16:10.863 INFO:teuthology.orchestra.run.smithi186.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-20T21:16:10.922 INFO:teuthology.orchestra.run.smithi186.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-20T21:16:10.972 INFO:teuthology.orchestra.run.smithi186.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-20T21:16:11.156 INFO:teuthology.orchestra.run.smithi186.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 32 MB/s | 5.8 MB 00:00 2024-04-20T21:16:11.223 INFO:teuthology.orchestra.run.smithi186.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-20T21:16:11.273 INFO:teuthology.orchestra.run.smithi186.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-20T21:16:11.431 INFO:teuthology.orchestra.run.smithi186.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-20T21:16:11.482 INFO:teuthology.orchestra.run.smithi186.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-20T21:16:11.532 INFO:teuthology.orchestra.run.smithi186.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-20T21:16:11.582 INFO:teuthology.orchestra.run.smithi186.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-20T21:16:11.633 INFO:teuthology.orchestra.run.smithi186.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-20T21:16:11.683 INFO:teuthology.orchestra.run.smithi186.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-20T21:16:11.720 INFO:teuthology.orchestra.run.smithi007.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 290 kB/s | 3.1 MB 00:10 2024-04-20T21:16:11.750 INFO:teuthology.orchestra.run.smithi186.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 522 kB/s | 35 kB 00:00 2024-04-20T21:16:11.925 INFO:teuthology.orchestra.run.smithi186.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-20T21:16:11.975 INFO:teuthology.orchestra.run.smithi186.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-20T21:16:12.026 INFO:teuthology.orchestra.run.smithi186.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-20T21:16:12.076 INFO:teuthology.orchestra.run.smithi186.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-20T21:16:12.126 INFO:teuthology.orchestra.run.smithi186.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-20T21:16:12.185 INFO:teuthology.orchestra.run.smithi186.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-20T21:16:12.235 INFO:teuthology.orchestra.run.smithi186.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-20T21:16:12.285 INFO:teuthology.orchestra.run.smithi186.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-20T21:16:12.312 INFO:teuthology.orchestra.run.smithi007.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 288 kB/s | 170 kB 00:00 2024-04-20T21:16:12.335 INFO:teuthology.orchestra.run.smithi186.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-20T21:16:12.386 INFO:teuthology.orchestra.run.smithi186.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-20T21:16:12.604 INFO:teuthology.orchestra.run.smithi007.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 118 kB/s | 34 kB 00:00 2024-04-20T21:16:12.903 INFO:teuthology.orchestra.run.smithi186.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-20T21:16:12.953 INFO:teuthology.orchestra.run.smithi186.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-04-20T21:16:13.012 INFO:teuthology.orchestra.run.smithi186.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-20T21:16:13.071 INFO:teuthology.orchestra.run.smithi186.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T21:16:13.129 INFO:teuthology.orchestra.run.smithi186.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-20T21:16:13.170 INFO:teuthology.orchestra.run.smithi007.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 251 kB/s | 142 kB 00:00 2024-04-20T21:16:13.179 INFO:teuthology.orchestra.run.smithi186.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-20T21:16:13.230 INFO:teuthology.orchestra.run.smithi186.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-20T21:16:13.288 INFO:teuthology.orchestra.run.smithi186.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-20T21:16:13.339 INFO:teuthology.orchestra.run.smithi186.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-20T21:16:13.389 INFO:teuthology.orchestra.run.smithi186.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-20T21:16:13.447 INFO:teuthology.orchestra.run.smithi186.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-20T21:16:13.854 INFO:teuthology.orchestra.run.smithi007.stdout:(30/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 283 kB/s | 5.4 MB 00:19 2024-04-20T21:16:14.039 INFO:teuthology.orchestra.run.smithi186.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-04-20T21:16:15.823 INFO:teuthology.orchestra.run.smithi186.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 484 kB/s | 863 kB 00:01 2024-04-20T21:16:16.090 INFO:teuthology.orchestra.run.smithi186.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-20T21:16:16.157 INFO:teuthology.orchestra.run.smithi186.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-04-20T21:16:16.207 INFO:teuthology.orchestra.run.smithi186.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 390 kB/s | 19 kB 00:00 2024-04-20T21:16:16.441 INFO:teuthology.orchestra.run.smithi186.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-20T21:16:16.491 INFO:teuthology.orchestra.run.smithi186.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-04-20T21:16:16.541 INFO:teuthology.orchestra.run.smithi186.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-20T21:16:16.583 INFO:teuthology.orchestra.run.smithi186.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-20T21:16:16.642 INFO:teuthology.orchestra.run.smithi186.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T21:16:16.709 INFO:teuthology.orchestra.run.smithi186.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-20T21:16:16.776 INFO:teuthology.orchestra.run.smithi186.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-20T21:16:16.826 INFO:teuthology.orchestra.run.smithi186.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 871 kB/s | 43 kB 00:00 2024-04-20T21:16:16.876 INFO:teuthology.orchestra.run.smithi186.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 645 kB/s | 32 kB 00:00 2024-04-20T21:16:16.927 INFO:teuthology.orchestra.run.smithi186.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-20T21:16:16.985 INFO:teuthology.orchestra.run.smithi186.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T21:16:17.044 INFO:teuthology.orchestra.run.smithi186.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-20T21:16:17.102 INFO:teuthology.orchestra.run.smithi186.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-20T21:16:17.145 INFO:teuthology.orchestra.run.smithi186.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 836 kB/s | 35 kB 00:00 2024-04-20T21:16:17.203 INFO:teuthology.orchestra.run.smithi186.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T21:16:17.295 INFO:teuthology.orchestra.run.smithi186.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-20T21:16:17.354 INFO:teuthology.orchestra.run.smithi186.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-20T21:16:17.395 INFO:teuthology.orchestra.run.smithi186.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-04-20T21:16:17.437 INFO:teuthology.orchestra.run.smithi186.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-04-20T21:16:17.488 INFO:teuthology.orchestra.run.smithi186.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-04-20T21:16:17.538 INFO:teuthology.orchestra.run.smithi186.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-20T21:16:17.589 INFO:teuthology.orchestra.run.smithi186.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 394 kB/s | 20 kB 00:00 2024-04-20T21:16:17.664 INFO:teuthology.orchestra.run.smithi186.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-20T21:16:17.714 INFO:teuthology.orchestra.run.smithi186.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 926 kB/s | 46 kB 00:00 2024-04-20T21:16:17.773 INFO:teuthology.orchestra.run.smithi186.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T21:16:17.823 INFO:teuthology.orchestra.run.smithi186.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-20T21:16:17.881 INFO:teuthology.orchestra.run.smithi186.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-20T21:16:17.932 INFO:teuthology.orchestra.run.smithi186.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-04-20T21:16:17.990 INFO:teuthology.orchestra.run.smithi186.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T21:16:18.040 INFO:teuthology.orchestra.run.smithi186.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-04-20T21:16:18.091 INFO:teuthology.orchestra.run.smithi186.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-04-20T21:16:18.149 INFO:teuthology.orchestra.run.smithi186.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T21:16:18.208 INFO:teuthology.orchestra.run.smithi186.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-20T21:16:18.266 INFO:teuthology.orchestra.run.smithi186.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 615 kB/s | 36 kB 00:00 2024-04-20T21:16:18.325 INFO:teuthology.orchestra.run.smithi186.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 875 kB/s | 51 kB 00:00 2024-04-20T21:16:18.384 INFO:teuthology.orchestra.run.smithi186.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-20T21:16:18.434 INFO:teuthology.orchestra.run.smithi186.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-20T21:16:18.492 INFO:teuthology.orchestra.run.smithi186.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-20T21:16:18.543 INFO:teuthology.orchestra.run.smithi186.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-20T21:16:18.602 INFO:teuthology.orchestra.run.smithi186.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-20T21:16:18.677 INFO:teuthology.orchestra.run.smithi186.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-20T21:16:21.346 INFO:teuthology.orchestra.run.smithi007.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 443 kB/s | 3.5 MB 00:08 2024-04-20T21:16:21.835 INFO:teuthology.orchestra.run.smithi186.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 343 kB/s | 7.4 MB 00:22 2024-04-20T21:16:22.220 INFO:teuthology.orchestra.run.smithi007.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 285 kB/s | 249 kB 00:00 2024-04-20T21:16:22.437 INFO:teuthology.orchestra.run.smithi007.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 229 kB/s | 49 kB 00:00 2024-04-20T21:16:22.596 INFO:teuthology.orchestra.run.smithi007.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 104 kB/s | 16 kB 00:00 2024-04-20T21:16:23.577 INFO:teuthology.orchestra.run.smithi186.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 710 kB/s | 3.4 MB 00:04 2024-04-20T21:16:23.579 INFO:teuthology.orchestra.run.smithi007.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 278 kB/s | 273 kB 00:00 2024-04-20T21:16:26.167 INFO:teuthology.orchestra.run.smithi016.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 361 kB/s | 17 MB 00:47 2024-04-20T21:16:26.519 INFO:teuthology.orchestra.run.smithi186.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 687 kB/s | 3.1 MB 00:04 2024-04-20T21:16:26.779 INFO:teuthology.orchestra.run.smithi007.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 238 kB/s | 763 kB 00:03 2024-04-20T21:16:26.793 INFO:teuthology.orchestra.run.smithi016.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 359 kB/s | 225 kB 00:00 2024-04-20T21:16:26.905 INFO:teuthology.orchestra.run.smithi007.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 147 kB/s | 18 kB 00:00 2024-04-20T21:16:26.918 INFO:teuthology.orchestra.run.smithi016.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 216 kB/s | 27 kB 00:00 2024-04-20T21:16:27.035 INFO:teuthology.orchestra.run.smithi016.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 272 kB/s | 32 kB 00:00 2024-04-20T21:16:27.096 INFO:teuthology.orchestra.run.smithi007.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 421 kB/s | 80 kB 00:00 2024-04-20T21:16:27.338 INFO:teuthology.orchestra.run.smithi007.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2024-04-20T21:16:27.430 INFO:teuthology.orchestra.run.smithi007.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-04-20T21:16:27.531 INFO:teuthology.orchestra.run.smithi007.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-20T21:16:27.781 INFO:teuthology.orchestra.run.smithi007.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-20T21:16:27.840 INFO:teuthology.orchestra.run.smithi007.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T21:16:27.898 INFO:teuthology.orchestra.run.smithi007.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-20T21:16:27.965 INFO:teuthology.orchestra.run.smithi007.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-20T21:16:28.032 INFO:teuthology.orchestra.run.smithi007.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-20T21:16:28.149 INFO:teuthology.orchestra.run.smithi007.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-20T21:16:28.258 INFO:teuthology.orchestra.run.smithi007.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 8.6 MB/s | 943 kB 00:00 2024-04-20T21:16:28.342 INFO:teuthology.orchestra.run.smithi007.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-04-20T21:16:28.400 INFO:teuthology.orchestra.run.smithi007.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 373 kB/s | 21 kB 00:00 2024-04-20T21:16:28.476 INFO:teuthology.orchestra.run.smithi016.stdout:(14/150): ceph-common-19.0.0-3133.ga8c84f59.el9 362 kB/s | 21 MB 00:59 2024-04-20T21:16:28.484 INFO:teuthology.orchestra.run.smithi007.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-20T21:16:28.543 INFO:teuthology.orchestra.run.smithi007.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 570 kB/s | 33 kB 00:00 2024-04-20T21:16:28.726 INFO:teuthology.orchestra.run.smithi007.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-20T21:16:28.777 INFO:teuthology.orchestra.run.smithi007.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 370 kB/s | 18 kB 00:00 2024-04-20T21:16:28.819 INFO:teuthology.orchestra.run.smithi007.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-04-20T21:16:28.886 INFO:teuthology.orchestra.run.smithi007.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-20T21:16:28.944 INFO:teuthology.orchestra.run.smithi007.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 790 kB/s | 45 kB 00:00 2024-04-20T21:16:29.045 INFO:teuthology.orchestra.run.smithi007.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-20T21:16:29.069 INFO:teuthology.orchestra.run.smithi016.stdout:(15/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 368 kB/s | 748 kB 00:02 2024-04-20T21:16:29.136 INFO:teuthology.orchestra.run.smithi007.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 6.3 MB/s | 582 kB 00:00 2024-04-20T21:16:29.187 INFO:teuthology.orchestra.run.smithi007.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 522 kB/s | 25 kB 00:00 2024-04-20T21:16:29.245 INFO:teuthology.orchestra.run.smithi007.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 828 kB/s | 47 kB 00:00 2024-04-20T21:16:29.329 INFO:teuthology.orchestra.run.smithi007.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-04-20T21:16:29.404 INFO:teuthology.orchestra.run.smithi007.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-20T21:16:29.455 INFO:teuthology.orchestra.run.smithi007.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 711 kB/s | 35 kB 00:00 2024-04-20T21:16:29.468 INFO:teuthology.orchestra.run.smithi016.stdout:(16/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 165 kB/s | 163 kB 00:00 2024-04-20T21:16:29.594 INFO:teuthology.orchestra.run.smithi016.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 241 kB/s | 126 kB 00:00 2024-04-20T21:16:29.714 INFO:teuthology.orchestra.run.smithi007.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-20T21:16:29.972 INFO:teuthology.orchestra.run.smithi007.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 16 MB/s | 4.1 MB 00:00 2024-04-20T21:16:30.114 INFO:teuthology.orchestra.run.smithi007.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-20T21:16:30.231 INFO:teuthology.orchestra.run.smithi007.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-04-20T21:16:30.306 INFO:teuthology.orchestra.run.smithi007.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.8 MB/s | 133 kB 00:00 2024-04-20T21:16:30.740 INFO:teuthology.orchestra.run.smithi007.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-20T21:16:30.857 INFO:teuthology.orchestra.run.smithi007.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-20T21:16:30.915 INFO:teuthology.orchestra.run.smithi007.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T21:16:31.291 INFO:teuthology.orchestra.run.smithi007.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-20T21:16:31.335 INFO:teuthology.orchestra.run.smithi016.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 268 kB/s | 501 kB 00:01 2024-04-20T21:16:31.374 INFO:teuthology.orchestra.run.smithi007.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-20T21:16:31.458 INFO:teuthology.orchestra.run.smithi007.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-20T21:16:31.525 INFO:teuthology.orchestra.run.smithi007.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 389 kB/s | 26 kB 00:00 2024-04-20T21:16:31.609 INFO:teuthology.orchestra.run.smithi007.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 232 kB/s | 19 kB 00:00 2024-04-20T21:16:31.684 INFO:teuthology.orchestra.run.smithi007.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-20T21:16:31.751 INFO:teuthology.orchestra.run.smithi007.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 526 kB/s | 35 kB 00:00 2024-04-20T21:16:31.761 INFO:teuthology.orchestra.run.smithi016.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 106 kB/s | 45 kB 00:00 2024-04-20T21:16:32.060 INFO:teuthology.orchestra.run.smithi007.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-20T21:16:32.152 INFO:teuthology.orchestra.run.smithi007.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-20T21:16:32.227 INFO:teuthology.orchestra.run.smithi007.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-20T21:16:32.294 INFO:teuthology.orchestra.run.smithi007.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-20T21:16:32.361 INFO:teuthology.orchestra.run.smithi007.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 640 kB/s | 42 kB 00:00 2024-04-20T21:16:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 200 kB/s | 135 kB 00:00 2024-04-20T21:16:32.478 INFO:teuthology.orchestra.run.smithi007.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-20T21:16:32.562 INFO:teuthology.orchestra.run.smithi007.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-20T21:16:32.704 INFO:teuthology.orchestra.run.smithi007.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-20T21:16:32.812 INFO:teuthology.orchestra.run.smithi007.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 475 kB/s | 51 kB 00:00 2024-04-20T21:16:32.921 INFO:teuthology.orchestra.run.smithi007.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 499 kB/s | 54 kB 00:00 2024-04-20T21:16:33.203 INFO:teuthology.orchestra.run.smithi016.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 211 kB/s | 162 kB 00:00 2024-04-20T21:16:34.353 INFO:teuthology.orchestra.run.smithi016.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 279 kB/s | 321 kB 00:01 2024-04-20T21:16:34.421 INFO:teuthology.orchestra.run.smithi007.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-20T21:16:34.488 INFO:teuthology.orchestra.run.smithi007.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 638 kB/s | 42 kB 00:00 2024-04-20T21:16:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 673 kB/s | 44 kB 00:00 2024-04-20T21:16:34.647 INFO:teuthology.orchestra.run.smithi007.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-20T21:16:34.697 INFO:teuthology.orchestra.run.smithi007.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 734 kB/s | 36 kB 00:00 2024-04-20T21:16:34.756 INFO:teuthology.orchestra.run.smithi007.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 825 kB/s | 47 kB 00:00 2024-04-20T21:16:34.840 INFO:teuthology.orchestra.run.smithi007.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-20T21:16:34.898 INFO:teuthology.orchestra.run.smithi007.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-20T21:16:34.965 INFO:teuthology.orchestra.run.smithi007.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-20T21:16:35.057 INFO:teuthology.orchestra.run.smithi007.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 702 kB/s | 64 kB 00:00 2024-04-20T21:16:35.116 INFO:teuthology.orchestra.run.smithi007.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 781 kB/s | 45 kB 00:00 2024-04-20T21:16:35.217 INFO:teuthology.orchestra.run.smithi007.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-04-20T21:16:35.253 INFO:teuthology.orchestra.run.smithi016.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 332 kB/s | 299 kB 00:00 2024-04-20T21:16:35.392 INFO:teuthology.orchestra.run.smithi007.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-20T21:16:35.609 INFO:teuthology.orchestra.run.smithi007.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-04-20T21:16:35.653 INFO:teuthology.orchestra.run.smithi016.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 249 kB/s | 99 kB 00:00 2024-04-20T21:16:35.667 INFO:teuthology.orchestra.run.smithi007.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 5.3 MB/s | 309 kB 00:00 2024-04-20T21:16:35.718 INFO:teuthology.orchestra.run.smithi007.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2024-04-20T21:16:35.926 INFO:teuthology.orchestra.run.smithi007.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-20T21:16:35.962 INFO:teuthology.orchestra.run.smithi016.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 276 kB/s | 85 kB 00:00 2024-04-20T21:16:35.985 INFO:teuthology.orchestra.run.smithi007.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 453 kB/s | 26 kB 00:00 2024-04-20T21:16:36.044 INFO:teuthology.orchestra.run.smithi007.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 844 kB/s | 49 kB 00:00 2024-04-20T21:16:36.086 INFO:teuthology.orchestra.run.smithi007.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-20T21:16:36.136 INFO:teuthology.orchestra.run.smithi007.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-20T21:16:36.228 INFO:teuthology.orchestra.run.smithi007.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.0 MB/s | 838 kB 00:00 2024-04-20T21:16:36.295 INFO:teuthology.orchestra.run.smithi007.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 8.2 MB/s | 548 kB 00:00 2024-04-20T21:16:36.337 INFO:teuthology.orchestra.run.smithi007.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-20T21:16:36.470 INFO:teuthology.orchestra.run.smithi007.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2024-04-20T21:16:36.512 INFO:teuthology.orchestra.run.smithi007.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 366 kB/s | 15 kB 00:00 2024-04-20T21:16:36.563 INFO:teuthology.orchestra.run.smithi007.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-20T21:16:36.621 INFO:teuthology.orchestra.run.smithi007.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2024-04-20T21:16:36.680 INFO:teuthology.orchestra.run.smithi007.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.2 MB/s | 303 kB 00:00 2024-04-20T21:16:36.730 INFO:teuthology.orchestra.run.smithi007.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 717 kB/s | 35 kB 00:00 2024-04-20T21:16:36.789 INFO:teuthology.orchestra.run.smithi007.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 3.5 MB/s | 203 kB 00:00 2024-04-20T21:16:36.906 INFO:teuthology.orchestra.run.smithi007.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 18 MB/s | 2.0 MB 00:00 2024-04-20T21:16:36.956 INFO:teuthology.orchestra.run.smithi007.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-04-20T21:16:37.006 INFO:teuthology.orchestra.run.smithi007.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 217 kB/s | 11 kB 00:00 2024-04-20T21:16:37.057 INFO:teuthology.orchestra.run.smithi007.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 360 kB/s | 18 kB 00:00 2024-04-20T21:16:37.107 INFO:teuthology.orchestra.run.smithi007.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2024-04-20T21:16:37.157 INFO:teuthology.orchestra.run.smithi007.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 396 kB/s | 19 kB 00:00 2024-04-20T21:16:37.208 INFO:teuthology.orchestra.run.smithi007.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2024-04-20T21:16:37.292 INFO:teuthology.orchestra.run.smithi007.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-20T21:16:37.367 INFO:teuthology.orchestra.run.smithi007.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 622 kB/s | 46 kB 00:00 2024-04-20T21:16:37.417 INFO:teuthology.orchestra.run.smithi007.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-20T21:16:37.468 INFO:teuthology.orchestra.run.smithi007.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-20T21:16:37.526 INFO:teuthology.orchestra.run.smithi007.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-20T21:16:37.576 INFO:teuthology.orchestra.run.smithi007.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 334 kB/s | 16 kB 00:00 2024-04-20T21:16:37.627 INFO:teuthology.orchestra.run.smithi007.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-20T21:16:37.760 INFO:teuthology.orchestra.run.smithi007.stdout:(135/150): ceph-mgr-diskprediction-local-19.0.0 317 kB/s | 7.4 MB 00:23 2024-04-20T21:16:37.786 INFO:teuthology.orchestra.run.smithi007.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 179 kB/s | 28 kB 00:00 2024-04-20T21:16:37.844 INFO:teuthology.orchestra.run.smithi007.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T21:16:37.870 INFO:teuthology.orchestra.run.smithi007.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 286 kB/s | 31 kB 00:00 2024-04-20T21:16:37.895 INFO:teuthology.orchestra.run.smithi007.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-20T21:16:37.945 INFO:teuthology.orchestra.run.smithi007.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 480 kB/s | 36 kB 00:00 2024-04-20T21:16:37.970 INFO:teuthology.orchestra.run.smithi007.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 687 kB/s | 51 kB 00:00 2024-04-20T21:16:38.021 INFO:teuthology.orchestra.run.smithi007.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-20T21:16:38.046 INFO:teuthology.orchestra.run.smithi007.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-20T21:16:38.079 INFO:teuthology.orchestra.run.smithi007.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-20T21:16:38.105 INFO:teuthology.orchestra.run.smithi007.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-04-20T21:16:38.147 INFO:teuthology.orchestra.run.smithi007.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-20T21:16:38.272 INFO:teuthology.orchestra.run.smithi007.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-20T21:16:45.286 INFO:teuthology.orchestra.run.smithi016.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 351 kB/s | 5.4 MB 00:15 2024-04-20T21:16:45.336 INFO:teuthology.orchestra.run.smithi016.stdout:(27/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 338 kB/s | 3.1 MB 00:09 2024-04-20T21:16:45.553 INFO:teuthology.orchestra.run.smithi016.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 158 kB/s | 34 kB 00:00 2024-04-20T21:16:45.770 INFO:teuthology.orchestra.run.smithi016.stdout:(29/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 352 kB/s | 170 kB 00:00 2024-04-20T21:16:46.329 INFO:teuthology.orchestra.run.smithi016.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 183 kB/s | 142 kB 00:00 2024-04-20T21:16:46.559 INFO:teuthology.orchestra.run.smithi007.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 388 kB/s | 3.1 MB 00:08 2024-04-20T21:16:49.979 INFO:teuthology.orchestra.run.smithi007.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 294 kB/s | 3.4 MB 00:11 2024-04-20T21:16:54.245 INFO:teuthology.orchestra.run.smithi016.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 428 kB/s | 3.5 MB 00:08 2024-04-20T21:16:55.203 INFO:teuthology.orchestra.run.smithi016.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 260 kB/s | 249 kB 00:00 2024-04-20T21:16:55.445 INFO:teuthology.orchestra.run.smithi016.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 204 kB/s | 49 kB 00:00 2024-04-20T21:16:55.579 INFO:teuthology.orchestra.run.smithi016.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 123 kB/s | 16 kB 00:00 2024-04-20T21:16:56.462 INFO:teuthology.orchestra.run.smithi016.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 309 kB/s | 273 kB 00:00 2024-04-20T21:16:58.171 INFO:teuthology.orchestra.run.smithi016.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 446 kB/s | 763 kB 00:01 2024-04-20T21:16:58.279 INFO:teuthology.orchestra.run.smithi016.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 168 kB/s | 18 kB 00:00 2024-04-20T21:16:58.396 INFO:teuthology.orchestra.run.smithi016.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 690 kB/s | 80 kB 00:00 2024-04-20T21:16:58.497 INFO:teuthology.orchestra.run.smithi016.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-04-20T21:16:58.530 INFO:teuthology.orchestra.run.smithi016.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 195 kB 00:00 2024-04-20T21:16:58.572 INFO:teuthology.orchestra.run.smithi016.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-20T21:16:58.639 INFO:teuthology.orchestra.run.smithi016.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-20T21:16:58.673 INFO:teuthology.orchestra.run.smithi016.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-20T21:16:58.715 INFO:teuthology.orchestra.run.smithi016.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-20T21:16:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-20T21:16:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-20T21:16:58.974 INFO:teuthology.orchestra.run.smithi016.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-04-20T21:16:59.050 INFO:teuthology.orchestra.run.smithi016.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-20T21:16:59.083 INFO:teuthology.orchestra.run.smithi016.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-20T21:16:59.117 INFO:teuthology.orchestra.run.smithi016.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 647 kB/s | 21 kB 00:00 2024-04-20T21:16:59.175 INFO:teuthology.orchestra.run.smithi016.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-20T21:16:59.209 INFO:teuthology.orchestra.run.smithi016.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 985 kB/s | 33 kB 00:00 2024-04-20T21:16:59.310 INFO:teuthology.orchestra.run.smithi016.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-20T21:16:59.343 INFO:teuthology.orchestra.run.smithi016.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-04-20T21:16:59.377 INFO:teuthology.orchestra.run.smithi016.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-04-20T21:16:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-20T21:16:59.460 INFO:teuthology.orchestra.run.smithi016.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-20T21:16:59.511 INFO:teuthology.orchestra.run.smithi016.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-20T21:16:59.569 INFO:teuthology.orchestra.run.smithi016.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-20T21:16:59.611 INFO:teuthology.orchestra.run.smithi016.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 614 kB/s | 25 kB 00:00 2024-04-20T21:16:59.653 INFO:teuthology.orchestra.run.smithi016.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-20T21:16:59.704 INFO:teuthology.orchestra.run.smithi016.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-20T21:16:59.754 INFO:teuthology.orchestra.run.smithi016.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-20T21:16:59.796 INFO:teuthology.orchestra.run.smithi016.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 852 kB/s | 35 kB 00:00 2024-04-20T21:16:59.929 INFO:teuthology.orchestra.run.smithi016.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2024-04-20T21:17:00.080 INFO:teuthology.orchestra.run.smithi016.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 28 MB/s | 4.1 MB 00:00 2024-04-20T21:17:00.155 INFO:teuthology.orchestra.run.smithi016.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-20T21:17:00.222 INFO:teuthology.orchestra.run.smithi016.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-20T21:17:00.264 INFO:teuthology.orchestra.run.smithi016.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-04-20T21:17:00.414 INFO:teuthology.orchestra.run.smithi016.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 39 MB/s | 5.8 MB 00:00 2024-04-20T21:17:00.481 INFO:teuthology.orchestra.run.smithi016.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-20T21:17:00.523 INFO:teuthology.orchestra.run.smithi016.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.0 MB/s | 87 kB 00:00 2024-04-20T21:17:00.682 INFO:teuthology.orchestra.run.smithi016.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-20T21:17:00.732 INFO:teuthology.orchestra.run.smithi016.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-20T21:17:00.782 INFO:teuthology.orchestra.run.smithi016.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-20T21:17:00.824 INFO:teuthology.orchestra.run.smithi016.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-04-20T21:17:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 462 kB/s | 19 kB 00:00 2024-04-20T21:17:00.908 INFO:teuthology.orchestra.run.smithi016.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-20T21:17:00.959 INFO:teuthology.orchestra.run.smithi016.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-04-20T21:17:01.101 INFO:teuthology.orchestra.run.smithi016.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-20T21:17:01.151 INFO:teuthology.orchestra.run.smithi016.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-20T21:17:01.201 INFO:teuthology.orchestra.run.smithi016.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-20T21:17:01.243 INFO:teuthology.orchestra.run.smithi016.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-20T21:17:01.310 INFO:teuthology.orchestra.run.smithi016.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-04-20T21:17:01.361 INFO:teuthology.orchestra.run.smithi016.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-20T21:17:01.402 INFO:teuthology.orchestra.run.smithi016.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-20T21:17:01.453 INFO:teuthology.orchestra.run.smithi016.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-20T21:17:01.495 INFO:teuthology.orchestra.run.smithi016.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-20T21:17:01.537 INFO:teuthology.orchestra.run.smithi016.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-20T21:17:01.904 INFO:teuthology.orchestra.run.smithi016.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-20T21:17:01.946 INFO:teuthology.orchestra.run.smithi016.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-20T21:17:01.979 INFO:teuthology.orchestra.run.smithi016.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-20T21:17:02.030 INFO:teuthology.orchestra.run.smithi016.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-20T21:17:02.063 INFO:teuthology.orchestra.run.smithi016.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-20T21:17:02.105 INFO:teuthology.orchestra.run.smithi016.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-20T21:17:02.155 INFO:teuthology.orchestra.run.smithi016.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-20T21:17:02.189 INFO:teuthology.orchestra.run.smithi016.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-04-20T21:17:02.231 INFO:teuthology.orchestra.run.smithi016.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-20T21:17:02.273 INFO:teuthology.orchestra.run.smithi016.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-20T21:17:02.332 INFO:teuthology.orchestra.run.smithi016.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-20T21:17:02.432 INFO:teuthology.orchestra.run.smithi016.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-04-20T21:17:02.757 INFO:teuthology.orchestra.run.smithi016.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-04-20T21:17:03.016 INFO:teuthology.orchestra.run.smithi016.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-04-20T21:17:03.091 INFO:teuthology.orchestra.run.smithi016.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-20T21:17:03.141 INFO:teuthology.orchestra.run.smithi016.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-20T21:17:03.325 INFO:teuthology.orchestra.run.smithi016.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-20T21:17:03.375 INFO:teuthology.orchestra.run.smithi016.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-04-20T21:17:03.442 INFO:teuthology.orchestra.run.smithi016.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-04-20T21:17:03.493 INFO:teuthology.orchestra.run.smithi016.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-20T21:17:03.551 INFO:teuthology.orchestra.run.smithi016.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-20T21:17:03.618 INFO:teuthology.orchestra.run.smithi016.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-20T21:17:03.719 INFO:teuthology.orchestra.run.smithi016.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-20T21:17:03.761 INFO:teuthology.orchestra.run.smithi016.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-20T21:17:03.803 INFO:teuthology.orchestra.run.smithi016.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 772 kB/s | 32 kB 00:00 2024-04-20T21:17:03.844 INFO:teuthology.orchestra.run.smithi016.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 360 kB/s | 15 kB 00:00 2024-04-20T21:17:03.903 INFO:teuthology.orchestra.run.smithi016.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-20T21:17:03.962 INFO:teuthology.orchestra.run.smithi016.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-20T21:17:04.021 INFO:teuthology.orchestra.run.smithi016.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-20T21:17:04.071 INFO:teuthology.orchestra.run.smithi016.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 709 kB/s | 35 kB 00:00 2024-04-20T21:17:04.129 INFO:teuthology.orchestra.run.smithi016.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T21:17:04.213 INFO:teuthology.orchestra.run.smithi016.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 25 MB/s | 2.0 MB 00:00 2024-04-20T21:17:04.272 INFO:teuthology.orchestra.run.smithi016.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-20T21:17:04.322 INFO:teuthology.orchestra.run.smithi016.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-20T21:17:04.372 INFO:teuthology.orchestra.run.smithi016.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-04-20T21:17:04.422 INFO:teuthology.orchestra.run.smithi016.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-04-20T21:17:04.473 INFO:teuthology.orchestra.run.smithi016.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-20T21:17:04.515 INFO:teuthology.orchestra.run.smithi016.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 474 kB/s | 20 kB 00:00 2024-04-20T21:17:04.582 INFO:teuthology.orchestra.run.smithi016.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-20T21:17:04.624 INFO:teuthology.orchestra.run.smithi016.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-20T21:17:04.683 INFO:teuthology.orchestra.run.smithi016.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-20T21:17:04.759 INFO:teuthology.orchestra.run.smithi016.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-20T21:17:04.800 INFO:teuthology.orchestra.run.smithi016.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-20T21:17:04.851 INFO:teuthology.orchestra.run.smithi016.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 330 kB/s | 16 kB 00:00 2024-04-20T21:17:04.909 INFO:teuthology.orchestra.run.smithi016.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-20T21:17:04.959 INFO:teuthology.orchestra.run.smithi016.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-04-20T21:17:05.010 INFO:teuthology.orchestra.run.smithi016.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-04-20T21:17:05.068 INFO:teuthology.orchestra.run.smithi016.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T21:17:05.119 INFO:teuthology.orchestra.run.smithi016.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-20T21:17:05.169 INFO:teuthology.orchestra.run.smithi016.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-04-20T21:17:05.220 INFO:teuthology.orchestra.run.smithi016.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-20T21:17:05.278 INFO:teuthology.orchestra.run.smithi016.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2024-04-20T21:17:05.337 INFO:teuthology.orchestra.run.smithi016.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-20T21:17:05.395 INFO:teuthology.orchestra.run.smithi016.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-04-20T21:17:05.445 INFO:teuthology.orchestra.run.smithi016.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-04-20T21:17:05.504 INFO:teuthology.orchestra.run.smithi016.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-20T21:17:05.579 INFO:teuthology.orchestra.run.smithi016.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-20T21:17:09.479 INFO:teuthology.orchestra.run.smithi016.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 327 kB/s | 7.4 MB 00:23 2024-04-20T21:17:16.187 INFO:teuthology.orchestra.run.smithi016.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 480 kB/s | 3.1 MB 00:06 2024-04-20T21:17:16.604 INFO:teuthology.orchestra.run.smithi016.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 315 kB/s | 3.4 MB 00:11 2024-04-20T21:17:17.934 INFO:teuthology.orchestra.run.smithi007.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 522 kB/s | 49 MB 01:36 2024-04-20T21:17:17.939 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:17:17.939 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.7 MB/s | 218 MB 02:09 2024-04-20T21:17:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-04-20T21:17:21.028 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-04-20T21:17:21.028 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-04-20T21:17:23.762 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-04-20T21:17:23.764 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-04-20T21:17:27.761 INFO:teuthology.orchestra.run.smithi186.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 468 kB/s | 49 MB 01:47 2024-04-20T21:17:27.765 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:17:27.765 INFO:teuthology.orchestra.run.smithi186.stdout:Total 1.5 MB/s | 218 MB 02:22 2024-04-20T21:17:28.766 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2024-04-20T21:17:28.893 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2024-04-20T21:17:28.894 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2024-04-20T21:17:30.062 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-04-20T21:17:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:17:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:17:30.387 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:17:30.617 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2024-04-20T21:17:30.617 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2024-04-20T21:17:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:17:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:31.171 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:31.372 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:31.413 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:17:31.450 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:17:31.890 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:17:31.961 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:17:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:32.154 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:32.178 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:32.285 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:32.312 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:32.376 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:32.819 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:32.879 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:33.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:17:33.213 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:17:33.257 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:17:33.308 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:17:33.414 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:17:33.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:17:33.524 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:17:33.575 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:17:33.615 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:17:33.739 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:17:33.813 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:17:33.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:17:33.948 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:17:34.069 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:17:34.103 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:17:34.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:17:34.193 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:17:34.320 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:17:34.391 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:17:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:17:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:17:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:17:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:17:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:17:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:17:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:17:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:17:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:17:35.866 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:17:35.970 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:17:36.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:17:36.059 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:17:36.137 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:17:36.199 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:17:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:17:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:17:36.483 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2024-04-20T21:17:36.543 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:17:36.555 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:17:36.580 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:17:36.596 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:17:36.623 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:17:36.633 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:17:36.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:17:36.833 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:17:36.873 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:17:36.929 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:17:36.960 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:17:36.964 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:17:37.006 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:17:37.064 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:37.186 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:37.199 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:17:37.215 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:37.307 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:37.351 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:37.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:17:37.383 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:17:37.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:17:37.429 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:17:37.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:17:37.521 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:17:37.560 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:17:37.602 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:17:37.611 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:17:37.631 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:17:37.726 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:17:37.735 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:37.757 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:37.776 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:17:37.819 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:37.830 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:17:37.841 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:37.864 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:37.881 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:17:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:17:38.016 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:17:38.135 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:38.172 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:17:38.174 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:38.238 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:17:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:17:38.294 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:17:38.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:17:38.390 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:17:38.423 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:17:38.459 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:17:38.512 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:17:38.534 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:17:38.574 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:17:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:17:38.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:17:38.650 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:17:38.678 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:17:38.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:17:38.834 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:17:38.864 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:17:38.908 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:17:38.986 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:17:39.019 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:17:39.062 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:17:39.090 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:17:39.119 INFO:teuthology.orchestra.run.smithi016.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 501 kB/s | 49 MB 01:40 2024-04-20T21:17:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:17:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:Total 1.7 MB/s | 218 MB 02:11 2024-04-20T21:17:39.162 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:17:39.214 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:17:39.290 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:17:39.430 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:17:39.482 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:17:39.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:17:39.656 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:17:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:17:39.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:17:39.894 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:17:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:17:40.017 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:17:40.049 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:17:40.096 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:17:40.124 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:17:40.147 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-04-20T21:17:40.175 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:17:40.213 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:17:40.246 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:17:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-04-20T21:17:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-04-20T21:17:40.289 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:17:40.301 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:17:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:17:40.374 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:17:40.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:17:40.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:17:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:17:40.488 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:17:40.524 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:17:40.608 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:17:40.609 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:17:40.747 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:17:40.790 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:17:40.820 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:17:40.862 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:17:40.956 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:17:40.986 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:17:41.026 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:17:41.052 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:17:41.077 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:17:41.151 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:17:41.278 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:17:41.317 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:17:41.345 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:17:41.361 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:17:41.379 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:17:41.423 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:17:41.461 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:17:41.492 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:17:41.523 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:17:41.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:17:41.557 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:17:41.590 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:17:41.632 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:17:41.658 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:17:41.737 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:17:41.788 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:17:41.819 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:17:41.867 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:17:41.966 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:17:42.013 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:17:42.114 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-04-20T21:17:42.115 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-04-20T21:17:42.742 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:17:42.785 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:17:42.819 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:17:42.918 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:17:42.976 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:17:43.063 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:17:43.103 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:17:43.280 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:17:43.287 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:17:43.310 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:17:43.387 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:17:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:17:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:17:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:17:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:17:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:44.042 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:17:44.065 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:44.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:17:44.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:17:44.176 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:17:44.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:17:44.786 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:17:45.423 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:17:45.521 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:17:45.549 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:17:45.582 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:17:45.893 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:17:45.937 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:46.028 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:46.058 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:17:46.094 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:17:46.572 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:17:46.615 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:46.686 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:46.708 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:17:47.882 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:47.934 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:48.120 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-04-20T21:17:48.196 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:17:48.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:17:48.282 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:17:48.608 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:48.626 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:17:48.654 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:48.739 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:48.863 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:17:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:48.877 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:17:48.877 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:17:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:17:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:48.928 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:17:48.963 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:48.982 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:17:49.024 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:17:49.053 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:17:49.078 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:17:49.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:17:49.293 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:17:49.322 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:49.411 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:17:49.440 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:49.486 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:17:49.506 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:17:49.507 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:49.531 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:17:49.558 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:49.653 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:17:49.654 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:17:49.654 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:17:49.654 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:17:49.692 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:17:49.811 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:49.840 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:17:49.952 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:17:50.050 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:17:50.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:17:50.125 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:17:50.201 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:17:50.241 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:17:50.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:17:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:17:50.344 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:17:50.418 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:17:50.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:17:50.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:17:50.549 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:17:50.628 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:17:50.660 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:17:50.704 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:17:50.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:17:50.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:17:50.863 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:17:50.961 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:17:51.108 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:17:51.170 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:17:51.755 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:17:51.831 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:17:51.866 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:17:51.909 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:17:51.947 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:17:51.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:17:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:17:52.112 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:17:52.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:17:52.184 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:17:52.232 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:17:52.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:17:52.310 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:17:52.354 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:17:52.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:17:52.541 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:17:52.571 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:17:52.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:17:52.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:17:52.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:17:52.786 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:17:52.819 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:17:52.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:17:52.923 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:17:53.053 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:17:53.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:17:53.113 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:17:53.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:17:53.192 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:17:53.229 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:17:53.269 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:17:53.298 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:17:53.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:17:53.366 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:17:53.409 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:17:53.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:17:53.533 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:17:53.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:17:53.612 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:17:53.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:17:53.781 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:17:53.832 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:17:54.608 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:17:54.661 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:17:54.695 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:17:54.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:17:54.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:17:54.944 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:17:54.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:17:55.164 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:17:55.186 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:17:55.257 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:17:55.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:17:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:17:57.409 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:17:57.498 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:17:57.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:17:57.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:17:57.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:17:57.930 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:58.021 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:17:58.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:17:58.592 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:17:58.625 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:58.696 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:17:58.726 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:18:00.891 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:00.891 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2024-04-20T21:18:00.891 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2024-04-20T21:18:00.891 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2024-04-20T21:18:00.892 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2024-04-20T21:18:00.892 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2024-04-20T21:18:00.892 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /root 2024-04-20T21:18:00.892 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2024-04-20T21:18:00.892 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:01.061 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:01.076 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:01.076 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:01.076 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:18:01.076 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:01.156 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:01.197 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:01.766 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:18:01.809 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:18:01.900 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:18:01.901 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:18:01.901 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:18:01.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:01.928 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:02.430 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-04-20T21:18:02.431 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:02.580 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:02.606 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:02.716 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:02.716 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:18:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:02.831 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:02.854 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:03.298 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:03.320 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:03.320 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:03.320 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:18:03.320 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:03.321 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:03.321 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:03.420 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:18:03.477 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:18:03.532 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:18:03.718 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:18:03.774 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:18:03.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:18:04.080 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:18:04.124 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:18:04.156 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:18:04.183 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:18:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:04.484 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:04.644 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:04.673 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:04.697 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:18:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:04.751 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:18:04.780 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:18:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:05.567 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:18:05.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:18:05.814 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:05.848 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:18:06.056 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:18:06.148 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:18:06.269 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:18:06.450 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:18:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:18:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:18:06.568 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:18:06.805 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:06.831 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:06.933 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:07.096 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:18:07.134 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:07.156 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:07.156 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:07.156 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:18:07.156 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:07.157 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:07.157 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:07.201 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:07.212 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:07.301 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:18:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:07.357 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:18:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:18:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:18:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:08.286 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:18:08.312 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:08.334 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:08.379 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:08.402 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:08.402 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:18:08.402 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:08.446 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:08.468 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:08.468 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:08.468 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:18:08.469 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:08.469 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:08.469 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:08.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:08.794 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:08.794 INFO:teuthology.orchestra.run.smithi186.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:08.794 INFO:teuthology.orchestra.run.smithi186.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:18:08.794 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:08.794 INFO:teuthology.orchestra.run.smithi186.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:08.795 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:08.993 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:10.154 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:10.184 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:10.340 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:10.557 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:18:10.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:10.737 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:10.780 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:10.916 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:18:10.997 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:18:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:18:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:12.366 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:12.366 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:18:12.366 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:12.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:12.542 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:12.542 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:12.542 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:18:12.543 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:12.543 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:12.543 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:12.992 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:12.993 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:18:12.993 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:12.993 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:13.350 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:18:13.391 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:18:13.433 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:18:13.465 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:18:13.492 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:18:13.673 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:18:13.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:18:13.743 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-04-20T21:18:13.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:14.047 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:18:14.063 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:14.063 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:18:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:14.357 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:14.415 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:14.435 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:15.218 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:18:15.311 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:18:15.351 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:18:15.434 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:18:15.465 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:18:15.465 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:15.532 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:15.532 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:15.592 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:18:15.619 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:15.851 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:15.874 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:18:15.874 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:15.874 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:18:15.874 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:15.875 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:18:15.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:16.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:18:16.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:18:16.524 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:18:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:18:16.789 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:18:16.845 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:18:16.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:18:17.015 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:18:17.165 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:18:17.203 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:18:17.235 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:18:17.262 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:18:17.674 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:17.703 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:18:17.736 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:18:17.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:18:17.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:18.282 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:18:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:18:18.419 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:18:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:18:18.545 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:18:18.888 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:18:18.920 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:18:18.953 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:18.954 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:18:18.981 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:18:19.892 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:19.938 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:19.966 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:18:19.986 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:20.005 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:20.094 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:18:20.265 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:18:20.305 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:18:20.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:20.381 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:20.392 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:18:20.502 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:18:20.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:18:21.394 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:18:21.420 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:18:21.421 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:18:21.422 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:18:21.423 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:18:21.424 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:18:21.425 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:18:21.426 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:18:21.427 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:18:21.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:18:21.429 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:18:21.430 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:18:21.431 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:18:21.503 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:18:21.528 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:18:21.559 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:21.582 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:18:21.582 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:21.582 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:18:21.582 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:21.583 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:18:21.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:21.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:18:21.626 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:21.648 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:18:21.649 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:21.649 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:18:21.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:21.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:18:21.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:21.792 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:18:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:18:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:22.044 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:22.067 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:18:22.067 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:18:22.067 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:18:22.067 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:22.068 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:18:22.068 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:22.545 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T21:18:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:18:23.460 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.461 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:18:23.462 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:18:23.463 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:18:23.464 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:18:23.465 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:18:23.466 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:18:23.467 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:18:23.468 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:18:23.469 INFO:teuthology.orchestra.run.smithi186.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:18:23.470 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:18:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:18:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:18:23.471 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:18:23.471 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2024-04-20T21:18:23.829 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:26.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:18:26.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:18:26.997 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:18:27.029 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:18:27.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:18:27.250 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:18:27.294 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:18:27.324 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:18:27.907 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:27.946 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:27.966 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:18:28.387 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:18:28.388 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:18:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:18:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:18:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:18:28.389 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:18:28.401 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:18:28.402 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:18:28.403 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:18:28.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:18:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:18:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:18:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:18:28.405 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:18:28.406 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:18:28.407 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:18:28.408 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:18:28.409 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:18:28.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:18:28.411 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:18:28.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:18:28.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:18:28.959 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:18:29.043 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:18:29.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:18:29.072 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:29.155 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:18:29.156 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.111 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.112 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:18:31.113 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:18:31.114 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:18:31.115 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.116 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:18:31.117 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:18:31.118 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:18:31.119 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:18:31.120 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:18:31.121 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:18:31.122 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-04-20T21:18:31.720 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:18:35.110 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:18:35.111 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:18:35.112 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:18:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:18:35.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:18:35.125 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:18:35.126 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:18:35.127 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:18:35.128 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:18:35.129 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:18:35.130 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:18:35.131 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:18:35.132 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:18:35.132 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:18:37.263 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:18:37.264 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.266 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:18:37.267 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.268 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:18:37.269 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.270 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:18:37.271 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:18:37.272 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:18:37.273 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:18:37.274 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:18:37.275 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:18:37.276 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:18:37.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-20T21:18:37.277 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-04-20T21:18:37.635 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:37.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:37.778 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:37.839 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:37.840 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:37.840 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:37.842 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:37.980 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:38.002 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:38.002 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:38.003 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:38.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:38.143 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:18:38.168 INFO:teuthology.orchestra.run.smithi186.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:38.168 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:18:38.168 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:18:38.170 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T21:18:38.171 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:18:38.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:38.244 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:18:38.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:38.271 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:18:38.271 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:18:38.302 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T21:18:38.302 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:18:38.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:38.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:38.464 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:18:38.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:38.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:38.553 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:18:38.553 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:18:38.582 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:18:38.649 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T21:18:38.649 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:18:38.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:38.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:38.849 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:18:38.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:38.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:38.937 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:18:38.937 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:18:38.963 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:18:39.028 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T21:18:39.028 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:18:39.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:39.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:39.215 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:18:39.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:39.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:39.303 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:18:39.303 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:18:39.326 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:18:39.390 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-20T21:18:39.486 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-20T21:18:39.486 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:39.486 INFO:tasks.cephadm:Cluster fsid is 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:18:39.487 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-20T21:18:39.487 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.7', 'mon.b': '172.21.15.16', 'mon.c': '172.21.15.186'} 2024-04-20T21:18:39.487 INFO:tasks.cephadm:First mon is mon.a on smithi007 2024-04-20T21:18:39.487 INFO:tasks.cephadm:First mgr is x 2024-04-20T21:18:39.487 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-20T21:18:39.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo hostname $(hostname -s) 2024-04-20T21:18:39.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-04-20T21:18:39.588 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2024-04-20T21:18:39.612 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-20T21:18:39.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:39.753 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78637/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3133-ga8c84f59', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3133.ga8c84f59'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-16 21:41:56.138585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-20T21:18:39.892 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735, sha1 a8c84f5997950fd7d008204839253291d52e6eda from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:18:39.894 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm 2024-04-20T21:18:39.894 DEBUG:teuthology.orchestra.run.smithi007:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:41.233 INFO:teuthology.orchestra.run.smithi007.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:41.234 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:42.393 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:42.393 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:43.022 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-r--r--. 1 ubuntu ubuntu 781646 Apr 20 21:18 /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:43.023 DEBUG:teuthology.orchestra.run.smithi007:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:43.089 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:43.108 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-20T21:18:43.143 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda on all hosts... 2024-04-20T21:18:43.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:43.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:43.153 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda pull 2024-04-20T21:18:44.821 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:18:44.886 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:18:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:19:35.443 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2024-04-20T21:19:35.444 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-20T21:20:06.826 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-04-20T21:20:06.827 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-20T21:21:20.211 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout: "image_id": "76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d", 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout: "repo_digests": [ 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1" 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:21:20.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:21:20.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph 2024-04-20T21:21:20.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-04-20T21:21:20.411 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2024-04-20T21:21:20.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /etc/ceph 2024-04-20T21:21:20.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-04-20T21:21:20.566 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2024-04-20T21:21:20.594 INFO:tasks.cephadm:Writing seed config... 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-20T21:21:20.595 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-20T21:21:20.596 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-20T21:21:20.597 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-20T21:21:20.598 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-20T21:21:20.599 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:21:20.599 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-20T21:21:20.659 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8e9da876-ff5b-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-20T21:21:20.659 DEBUG:teuthology.orchestra.run.smithi007:mon.a> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service 2024-04-20T21:21:20.707 DEBUG:teuthology.orchestra.run.smithi007:mgr.x> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x.service 2024-04-20T21:21:20.752 INFO:tasks.cephadm:Bootstrapping... 2024-04-20T21:21:20.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda -v bootstrap --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.7 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:21:21.037 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:21:21.037 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda', '-v', 'bootstrap', '--fsid', '8e9da876-ff5b-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.7', '--skip-admin-label'] 2024-04-20T21:21:21.037 INFO:teuthology.orchestra.run.smithi007.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-20T21:21:21.037 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying podman|docker is present... 2024-04-20T21:21:21.097 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T21:21:21.097 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying lvm2 is present... 2024-04-20T21:21:21.097 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying time synchronization is in place... 2024-04-20T21:21:21.113 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T21:21:21.113 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T21:21:21.129 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T21:21:21.129 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:21:21.148 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-04-20T21:21:21.168 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-04-20T21:21:21.168 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-04-20T21:21:21.169 INFO:teuthology.orchestra.run.smithi007.stdout:Repeating the final host check... 2024-04-20T21:21:21.234 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-20T21:21:21.234 INFO:teuthology.orchestra.run.smithi007.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-20T21:21:21.234 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl is present 2024-04-20T21:21:21.234 INFO:teuthology.orchestra.run.smithi007.stdout:lvcreate is present 2024-04-20T21:21:21.255 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-20T21:21:21.255 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-20T21:21:21.278 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-20T21:21:21.278 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:21:21.304 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout enabled 2024-04-20T21:21:21.327 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout active 2024-04-20T21:21:21.327 INFO:teuthology.orchestra.run.smithi007.stdout:Unit chronyd.service is enabled and running 2024-04-20T21:21:21.328 INFO:teuthology.orchestra.run.smithi007.stdout:Host looks OK 2024-04-20T21:21:21.328 INFO:teuthology.orchestra.run.smithi007.stdout:Cluster fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:21.328 INFO:teuthology.orchestra.run.smithi007.stdout:Acquiring lock 139955407346848 on /run/cephadm/8e9da876-ff5b-11ee-bc93-c7b262605968.lock 2024-04-20T21:21:21.329 INFO:teuthology.orchestra.run.smithi007.stdout:Lock 139955407346848 acquired on /run/cephadm/8e9da876-ff5b-11ee-bc93-c7b262605968.lock 2024-04-20T21:21:21.330 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 3300 ... 2024-04-20T21:21:21.331 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying IP 172.21.15.7 port 6789 ... 2024-04-20T21:21:21.332 INFO:teuthology.orchestra.run.smithi007.stdout:Base mon IP(s) is [172.21.15.7:3300, 172.21.15.7:6789], mon addrv is [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-04-20T21:21:21.346 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.7 metric 100 2024-04-20T21:21:21.347 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.7 metric 100 2024-04-20T21:21:21.361 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-20T21:21:21.361 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-20T21:21:21.361 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-20T21:21:21.372 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-20T21:21:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-04-20T21:21:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:Mon IP `172.21.15.7` is in CIDR network `172.21.0.0/20` 2024-04-20T21:21:21.375 INFO:teuthology.orchestra.run.smithi007.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-20T21:21:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-20T21:21:21.381 INFO:teuthology.orchestra.run.smithi007.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:21:22.368 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stdout 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda... 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Getting image source signatures 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:9473d3fd79377e8e2904ab0f292b04da81cb2f2bfccde15ec927b481a725f6f4 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Copying config sha256:76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d 2024-04-20T21:21:22.369 INFO:teuthology.orchestra.run.smithi007.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-20T21:21:23.588 INFO:teuthology.orchestra.run.smithi007.stdout:ceph: stdout ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:21:23.588 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph version: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev) 2024-04-20T21:21:23.589 INFO:teuthology.orchestra.run.smithi007.stdout:Extracting ceph user uid/gid from container image... 2024-04-20T21:21:24.683 INFO:teuthology.orchestra.run.smithi007.stdout:stat: stdout 167 167 2024-04-20T21:21:24.683 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial keys... 2024-04-20T21:21:26.126 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQDVMSRm9LQhGxAAlfTTcKO780dWLfkLWl3B1g== 2024-04-20T21:21:27.248 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQDWMSRmrU7IIxAA1RNHYYXUaimZKw7DR0y+Bw== 2024-04-20T21:21:28.434 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-authtool: stdout AQDXMSRmjTu9KhAAC+82AM8bbkZR1EWq+bTFag== 2024-04-20T21:21:28.438 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial monmap... 2024-04-20T21:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T21:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-20T21:21:29.806 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool for a [v2:172.21.15.7:3300,v1:172.21.15.7:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: set fsid to 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:21:29.807 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mon... 2024-04-20T21:21:38.830 INFO:teuthology.orchestra.run.smithi007.stdout:create mon.a on 2024-04-20T21:21:39.168 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-20T21:21:39.409 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-20T21:21:39.712 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target. 2024-04-20T21:21:39.712 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target. 2024-04-20T21:21:40.106 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a 2024-04-20T21:21:40.106 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-20T21:21:40.404 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target.wants/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@.service. 2024-04-20T21:21:41.110 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 podman[16482]: 2024-04-20 21:21:41.110226973 +0000 UTC m=+0.416007445 container init ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2) 2024-04-20T21:21:41.218 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:21:41.219 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-04-20T21:21:41.231 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:21:41.232 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:21:41.232 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-04-20T21:21:41.232 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T21:21:41.232 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon to start... 2024-04-20T21:21:41.232 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mon... 2024-04-20T21:21:41.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 podman[16482]: 2024-04-20 21:21:41.121558229 +0000 UTC m=+0.427338697 container start ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:21:41.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 bash[16482]: ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 2024-04-20T21:21:41.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 systemd[1]: Started Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:21:41.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 ceph-mon[16502]: mkfs 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:41.385 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:41 smithi007 ceph-mon[16502]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: monmap epoch 1 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: last_changed 2024-04-20T21:21:28.997163+0000 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:21:42.476 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: min_mon_release 19 (squid) 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: election_strategy: 1 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: fsmap 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: mgrmap e1: no daemons active 2024-04-20T21:21:42.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:42 smithi007 ceph-mon[16502]: from='client.? 172.21.15.7:0/3161582667' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-20T21:21:42.485 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout cluster: 2024-04-20T21:21:42.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout id: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:42.486 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-20T21:21:42.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:42.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout services: 2024-04-20T21:21:42.487 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.701359s) 2024-04-20T21:21:42.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-20T21:21:42.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-20T21:21:42.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:42.488 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout data: 2024-04-20T21:21:42.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-20T21:21:42.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-20T21:21:42.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-20T21:21:42.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout pgs: 2024-04-20T21:21:42.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:42.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.810+0000 7fc39e000700 1 Processor -- start 2024-04-20T21:21:42.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.810+0000 7fc39e000700 1 -- start start 2024-04-20T21:21:42.490 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.810+0000 7fc39e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:42.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.810+0000 7fc39e000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc398108ff0 con 0x7fc398108600 2024-04-20T21:21:42.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.811+0000 7fc397400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:42.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.811+0000 7fc397400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58980/0 (socket says 172.21.15.7:58980) 2024-04-20T21:21:42.491 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.811+0000 7fc397400700 1 -- 172.21.15.7:0/12105500 learned_addr learned my addr 172.21.15.7:0/12105500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:42.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.812+0000 7fc397400700 1 -- 172.21.15.7:0/12105500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc398109800 con 0x7fc398108600 2024-04-20T21:21:42.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.812+0000 7fc397400700 1 --2- 172.21.15.7:0/12105500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb2091d64248308 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:42.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.812+0000 7fc396000700 1 -- 172.21.15.7:0/12105500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388004160 con 0x7fc398108600 2024-04-20T21:21:42.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc396000700 1 -- 172.21.15.7:0/12105500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388004300 con 0x7fc398108600 2024-04-20T21:21:42.492 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc396000700 1 -- 172.21.15.7:0/12105500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3880036a0 con 0x7fc398108600 2024-04-20T21:21:42.493 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc39e000700 1 -- 172.21.15.7:0/12105500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 msgr2=0x7fc398108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:42.493 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc39e000700 1 --2- 172.21.15.7:0/12105500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098a0 tx=0x7fc388009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:42.493 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc39e000700 1 -- 172.21.15.7:0/12105500 shutdown_connections 2024-04-20T21:21:42.493 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc39e000700 1 --2- 172.21.15.7:0/12105500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc398108a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:42.493 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.813+0000 7fc39e000700 1 -- 172.21.15.7:0/12105500 >> 172.21.15.7:0/12105500 conn(0x7fc398103be0 msgr2=0x7fc398106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:42.494 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.814+0000 7fc39e000700 1 -- 172.21.15.7:0/12105500 shutdown_connections 2024-04-20T21:21:42.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.814+0000 7fc39e000700 1 -- 172.21.15.7:0/12105500 wait complete. 2024-04-20T21:21:42.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc39e000700 1 Processor -- start 2024-04-20T21:21:42.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc39e000700 1 -- start start 2024-04-20T21:21:42.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc39e000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:42.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc39e000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc398109d30 con 0x7fc398108600 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc397400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc397400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58988/0 (socket says 172.21.15.7:58988) 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.815+0000 7fc397400700 1 -- 172.21.15.7:0/3161582667 learned_addr learned my addr 172.21.15.7:0/3161582667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.816+0000 7fc397400700 1 -- 172.21.15.7:0/3161582667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39819e400 con 0x7fc398108600 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.816+0000 7fc397400700 1 --2- 172.21.15.7:0/3161582667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:42.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.816+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388003db0 con 0x7fc398108600 2024-04-20T21:21:42.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.816+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388011630 con 0x7fc398108600 2024-04-20T21:21:42.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.816+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39819e690 con 0x7fc398108600 2024-04-20T21:21:42.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.817+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc388011950 con 0x7fc398108600 2024-04-20T21:21:42.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.817+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981a1380 con 0x7fc398108600 2024-04-20T21:21:42.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.818+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc38801a460 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.818+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc38801b780 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.818+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3980640b0 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.821+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc388031020 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.902+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc39802e150 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.902+0000 7fc38fe00700 1 -- 172.21.15.7:0/3161582667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc38801b9d0 con 0x7fc398108600 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.903+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 msgr2=0x7fc39819dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.903+0000 7fc39e000700 1 --2- 172.21.15.7:0/3161582667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc388000c00 tx=0x7fc388004750 comp rx=0 tx=0).stop 2024-04-20T21:21:42.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.903+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 shutdown_connections 2024-04-20T21:21:42.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.903+0000 7fc39e000700 1 --2- 172.21.15.7:0/3161582667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc398108600 0x7fc39819dec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:42.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.903+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 >> 172.21.15.7:0/3161582667 conn(0x7fc398103be0 msgr2=0x7fc398194d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:42.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.904+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 shutdown_connections 2024-04-20T21:21:42.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:41.904+0000 7fc39e000700 1 -- 172.21.15.7:0/3161582667 wait complete. 2024-04-20T21:21:42.501 INFO:teuthology.orchestra.run.smithi007.stdout:mon is available 2024-04-20T21:21:42.501 INFO:teuthology.orchestra.run.smithi007.stdout:Assimilating anything we can from ceph.conf... 2024-04-20T21:21:43.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:43 smithi007 ceph-mon[16502]: from='client.? 172.21.15.7:0/408963567' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T21:21:43.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:43 smithi007 ceph-mon[16502]: from='client.? 172.21.15.7:0/408963567' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-20T21:21:44.048 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T21:21:44.049 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.7:3300,v1:172.21.15.7:6789] 2024-04-20T21:21:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T21:21:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T21:21:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T21:21:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T21:21:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-20T21:21:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.097+0000 7fc304c00700 1 Processor -- start 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.098+0000 7fc304c00700 1 -- start start 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.098+0000 7fc304c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.098+0000 7fc304c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc30010fa70 con 0x7fc30010f080 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.099+0000 7fc2fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.099+0000 7fc2fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33972/0 (socket says 172.21.15.7:33972) 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.099+0000 7fc2fde00700 1 -- 172.21.15.7:0/3205658191 learned_addr learned my addr 172.21.15.7:0/3205658191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.100+0000 7fc2fde00700 1 -- 172.21.15.7:0/3205658191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30010fe40 con 0x7fc30010f080 2024-04-20T21:21:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.100+0000 7fc2fde00700 1 --2- 172.21.15.7:0/3205658191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618e87a1229c43da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.100+0000 7fc2fca00700 1 -- 172.21.15.7:0/3205658191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0004160 con 0x7fc30010f080 2024-04-20T21:21:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.100+0000 7fc2fca00700 1 -- 172.21.15.7:0/3205658191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f0004300 con 0x7fc30010f080 2024-04-20T21:21:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.100+0000 7fc2fca00700 1 -- 172.21.15.7:0/3205658191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f00036a0 con 0x7fc30010f080 2024-04-20T21:21:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 -- 172.21.15.7:0/3205658191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 msgr2=0x7fc30010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 --2- 172.21.15.7:0/3205658191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098a0 tx=0x7fc2f0009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 -- 172.21.15.7:0/3205658191 shutdown_connections 2024-04-20T21:21:44.059 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 --2- 172.21.15.7:0/3205658191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30010f080 0x7fc30010f4a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 -- 172.21.15.7:0/3205658191 >> 172.21.15.7:0/3205658191 conn(0x7fc30007de30 msgr2=0x7fc300080270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 -- 172.21.15.7:0/3205658191 shutdown_connections 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.101+0000 7fc304c00700 1 -- 172.21.15.7:0/3205658191 wait complete. 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.102+0000 7fc304c00700 1 Processor -- start 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc304c00700 1 -- start start 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc304c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc304c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc300110370 con 0x7fc3001a2480 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc2fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc2fde00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33984/0 (socket says 172.21.15.7:33984) 2024-04-20T21:21:44.060 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.103+0000 7fc2fde00700 1 -- 172.21.15.7:0/408963567 learned_addr learned my addr 172.21.15.7:0/408963567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc2fde00700 1 -- 172.21.15.7:0/408963567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001a2de0 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc2fde00700 1 --2- 172.21.15.7:0/408963567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f0003ca0 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f0003e40 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001a3070 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2f00117a0 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.104+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a5b50 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.105+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc2f0011cd0 con 0x7fc3001a2480 2024-04-20T21:21:44.061 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.105+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc2f001a7a0 con 0x7fc3001a2480 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.106+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3000640b0 con 0x7fc3001a2480 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.109+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc2f0006e20 con 0x7fc3001a2480 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.187+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc30002e210 con 0x7fc3001a2480 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.196+0000 7fc2f6a00700 1 -- 172.21.15.7:0/408963567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+452 (secure 0 0 0) 0x7fc2f0024d30 con 0x7fc3001a2480 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.197+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 msgr2=0x7fc3001a28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.197+0000 7fc304c00700 1 --2- 172.21.15.7:0/408963567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0004750 comp rx=0 tx=0).stop 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.197+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 shutdown_connections 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.197+0000 7fc304c00700 1 --2- 172.21.15.7:0/408963567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3001a2480 0x7fc3001a28a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.197+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 >> 172.21.15.7:0/408963567 conn(0x7fc30007de30 msgr2=0x7fc300198ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.198+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 shutdown_connections 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:43.198+0000 7fc304c00700 1 -- 172.21.15.7:0/408963567 wait complete. 2024-04-20T21:21:44.062 INFO:teuthology.orchestra.run.smithi007.stdout:Generating new minimal ceph.conf... 2024-04-20T21:21:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:44 smithi007 ceph-mon[16502]: from='client.? 172.21.15.7:0/4263124336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:21:45.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.651+0000 7f85d5400700 1 Processor -- start 2024-04-20T21:21:45.380 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.651+0000 7f85d5400700 1 -- start start 2024-04-20T21:21:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.652+0000 7f85d5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.652+0000 7f85d5400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85d010b0e0 con 0x7f85d010a6f0 2024-04-20T21:21:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.652+0000 7f85cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.652+0000 7f85cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33998/0 (socket says 172.21.15.7:33998) 2024-04-20T21:21:45.381 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.652+0000 7f85cfe00700 1 -- 172.21.15.7:0/1020630325 learned_addr learned my addr 172.21.15.7:0/1020630325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.653+0000 7f85cfe00700 1 -- 172.21.15.7:0/1020630325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d010b850 con 0x7f85d010a6f0 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.653+0000 7f85cfe00700 1 --2- 172.21.15.7:0/1020630325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85c00098a0 tx=0x7f85c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c636056ff0693e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.654+0000 7f85cea00700 1 -- 172.21.15.7:0/1020630325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c0004160 con 0x7f85d010a6f0 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.654+0000 7f85cea00700 1 -- 172.21.15.7:0/1020630325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85c0004300 con 0x7f85d010a6f0 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.654+0000 7f85cea00700 1 -- 172.21.15.7:0/1020630325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c0011b50 con 0x7f85d010a6f0 2024-04-20T21:21:45.382 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.654+0000 7f85d5400700 1 -- 172.21.15.7:0/1020630325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 msgr2=0x7f85d010ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.654+0000 7f85d5400700 1 --2- 172.21.15.7:0/1020630325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f85c00098a0 tx=0x7f85c0009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.655+0000 7f85d5400700 1 -- 172.21.15.7:0/1020630325 shutdown_connections 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.655+0000 7f85d5400700 1 --2- 172.21.15.7:0/1020630325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d010ab10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.655+0000 7f85d5400700 1 -- 172.21.15.7:0/1020630325 >> 172.21.15.7:0/1020630325 conn(0x7f85d007de30 msgr2=0x7f85d01084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.655+0000 7f85d5400700 1 -- 172.21.15.7:0/1020630325 shutdown_connections 2024-04-20T21:21:45.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.655+0000 7f85d5400700 1 -- 172.21.15.7:0/1020630325 wait complete. 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.656+0000 7f85d5400700 1 Processor -- start 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.656+0000 7f85d5400700 1 -- start start 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85d5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85d5400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85d010ec00 con 0x7f85d010a6f0 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:45.385 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85cfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34008/0 (socket says 172.21.15.7:34008) 2024-04-20T21:21:45.387 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85cfe00700 1 -- 172.21.15.7:0/4263124336 learned_addr learned my addr 172.21.15.7:0/4263124336 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:45.387 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.657+0000 7f85cfe00700 1 -- 172.21.15.7:0/4263124336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d019eaf0 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85cfe00700 1 --2- 172.21.15.7:0/4263124336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f85c0011850 tx=0x7f85c0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c0003af0 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85c0020070 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c00195a0 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d019ed80 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01a1a70 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85c0019be0 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.658+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85c0011e00 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.660+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d00640b0 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.662+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f85c0003c90 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.744+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f85d01a1c40 con 0x7f85d010a6f0 2024-04-20T21:21:45.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.745+0000 7f85ccc00700 1 -- 172.21.15.7:0/4263124336 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f85c0025460 con 0x7f85d010a6f0 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.746+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 msgr2=0x7f85d019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.746+0000 7f85d5400700 1 --2- 172.21.15.7:0/4263124336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f85c0011850 tx=0x7f85c0011c10 comp rx=0 tx=0).stop 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.746+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 shutdown_connections 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.746+0000 7f85d5400700 1 --2- 172.21.15.7:0/4263124336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d010a6f0 0x7f85d019e5b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.746+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 >> 172.21.15.7:0/4263124336 conn(0x7f85d007de30 msgr2=0x7f85d0107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.747+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 shutdown_connections 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:44.747+0000 7f85d5400700 1 -- 172.21.15.7:0/4263124336 wait complete. 2024-04-20T21:21:45.389 INFO:teuthology.orchestra.run.smithi007.stdout:Restarting the monitor... 2024-04-20T21:21:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:45 smithi007 systemd[1]: Stopping Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:21:46.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a[16498]: 2024-04-20T21:21:46.048+0000 7f0d2dc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:21:46.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a[16498]: 2024-04-20T21:21:46.048+0000 7f0d2dc00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-20T21:21:46.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 podman[16743]: 2024-04-20 21:21:46.265394373 +0000 UTC m=+0.488202535 container died ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-20T21:21:46.925 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 podman[16743]: 2024-04-20 21:21:46.673560975 +0000 UTC m=+0.896369170 container cleanup ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:21:46.925 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 bash[16743]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a 2024-04-20T21:21:47.176 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:46 smithi007 podman[16757]: 2024-04-20 21:21:46.924245291 +0000 UTC m=+0.639948479 container remove ec80cc8ff3fcc275bbdaa297f681889f2957103f6a819e2ede231d2977dae566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T21:21:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 systemd[1]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-20T21:21:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 systemd[1]: Stopped Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:21:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 systemd[1]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service: Consumed 1.987s CPU time. 2024-04-20T21:21:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 systemd[1]: Starting Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:21:48.128 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 podman[16848]: 2024-04-20 21:21:47.739381754 +0000 UTC m=+0.035218600 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:21:48.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:47 smithi007 podman[16848]: 2024-04-20 21:21:47.867257836 +0000 UTC m=+0.163094708 container create 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:21:48.129 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 podman[16848]: 2024-04-20 21:21:48.129338136 +0000 UTC m=+0.425175014 container init 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-20T21:21:48.228 INFO:teuthology.orchestra.run.smithi007.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 podman[16848]: 2024-04-20 21:21:48.146857676 +0000 UTC m=+0.442694555 container start 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: pidfile_write: ignore empty --pid-file 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: load: jerasure load: lrc 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: RocksDB version: 7.9.2 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Git sha 0 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: DB SUMMARY 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: DB Session ID: PO1YMYRWT1EH96KWP1PP 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: CURRENT file: CURRENT 2024-04-20T21:21:48.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106090 ; 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.error_if_exists: 0 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.create_if_missing: 0 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.paranoid_checks: 1 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.env: 0x55da8a246cc0 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.info_log: 0x55daa70a1c60 2024-04-20T21:21:48.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.statistics: (nil) 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.use_fsync: 0 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_log_file_size: 0 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T21:21:48.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_fallocate: 1 2024-04-20T21:21:48.398 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.use_direct_reads: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.db_log_dir: 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.wal_dir: 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T21:21:48.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.write_buffer_manager: 0x55daa70701e0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.unordered_write: 0 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T21:21:48.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.row_cache: None 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.wal_filter: None 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.two_write_queues: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.wal_compression: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.atomic_flush: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.log_readahead_size: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T21:21:48.401 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_background_jobs: 2 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_background_compactions: -1 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_subcompactions: 1 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T21:21:48.402 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_open_files: -1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_background_flushes: -1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Compression algorithms supported: 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kZSTD supported: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kXpressCompression supported: 0 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kZlibCompression supported: 1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kSnappyCompression supported: 1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kLZ4Compression supported: 1 2024-04-20T21:21:48.403 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: kBZip2Compression supported: 0 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.merge_operator: 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_filter: None 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55daa70a1e20) 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks: 1 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T21:21:48.404 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: pin_top_level_index_and_filter: 1 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_type: 0 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_index_type: 0 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_shortening: 1 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: checksum: 4 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: no_block_cache: 0 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache: 0x55daa70a3090 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_name: BinnedLRUCache 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_options: 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: capacity : 536870912 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_shard_bits : 4 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: strict_capacity_limit : 0 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: high_pri_pool_ratio: 0.000 2024-04-20T21:21:48.405 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_cache_compressed: (nil) 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: persistent_cache: (nil) 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size: 4096 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_size_deviation: 10 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_restart_interval: 16 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: index_block_restart_interval: 1 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: metadata_block_size: 4096 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: partition_filters: 0 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: use_delta_encoding: 1 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: filter_policy: bloomfilter 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: whole_key_filtering: 1 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: verify_compression: 0 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: read_amp_bytes_per_bit: 0 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: format_version: 5 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_index_compression: 1 2024-04-20T21:21:48.406 INFO:journalctl@ceph.mon.a.smithi007.stdout: block_align: 0 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout: max_auto_readahead_size: 262144 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout: prepopulate_block_cache: 0 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout: initial_auto_readahead_size: 8192 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression: NoCompression 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.num_levels: 7 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T21:21:48.407 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T21:21:48.408 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T21:21:48.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T21:21:48.410 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.inplace_update_support: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.bloom_locality: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.max_successive_merges: 0 2024-04-20T21:21:48.411 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.ttl: 2592000 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enable_blob_files: false 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.min_blob_size: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T21:21:48.412 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2addbe0c-0138-4ad6-b618-dd59f735bb32 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648108205876, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648108207972, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98989, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 91192, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713648108, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2addbe0c-0138-4ad6-b618-dd59f735bb32", "db_session_id": "PO1YMYRWT1EH96KWP1PP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648108208160, "job": 1, "event": "recovery_finished"} 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55daa7190000 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: DB pointer 0x55daa717a000 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** DB Stats ** 2024-04-20T21:21:48.413 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.414 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** Compaction Stats [default] ** 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Cumulative compaction: 0.00 GB write, 13.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Interval compaction: 0.00 GB write, 13.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T21:21:48.415 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache BinnedLRUCache@0x55daa70a3090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: starting mon.a rank 0 at public addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] at bind addrs [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???) e1 preinit fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).mds e1 new map 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).mds e1 print_map 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: e1 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:21:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout: legacy client fscid: -1 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout: 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout: No filesystems configured 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 bash[16848]: 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 systemd[1]: Started Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: monmap epoch 1 2024-04-20T21:21:48.417 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: last_changed 2024-04-20T21:21:28.997163+0000 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: min_mon_release 19 (squid) 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: election_strategy: 1 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: fsmap 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: osdmap e1: 0 total, 0 up, 0 in 2024-04-20T21:21:48.418 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:48 smithi007 ceph-mon[16867]: mgrmap e1: no daemons active 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.989+0000 7f3109800700 1 Processor -- start 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.990+0000 7f3109800700 1 -- start start 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.990+0000 7f3109800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.990+0000 7f3109800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3104107120 con 0x7f3104106730 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.991+0000 7f3103e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.991+0000 7f3103e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34020/0 (socket says 172.21.15.7:34020) 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.991+0000 7f3103e00700 1 -- 172.21.15.7:0/528959967 learned_addr learned my addr 172.21.15.7:0/528959967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.992+0000 7f3103e00700 1 -- 172.21.15.7:0/528959967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31041096c0 con 0x7f3104106730 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.992+0000 7f3103e00700 1 --2- 172.21.15.7:0/528959967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30f40098a0 tx=0x7f30f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9579724a5d8241 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.992+0000 7f3102a00700 1 -- 172.21.15.7:0/528959967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f4004160 con 0x7f3104106730 2024-04-20T21:21:49.747 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.993+0000 7f3102a00700 1 -- 172.21.15.7:0/528959967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f30f4004300 con 0x7f3104106730 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.993+0000 7f3102a00700 1 -- 172.21.15.7:0/528959967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f4011b50 con 0x7f3104106730 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.993+0000 7f3109800700 1 -- 172.21.15.7:0/528959967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 msgr2=0x7f3104106b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.993+0000 7f3109800700 1 --2- 172.21.15.7:0/528959967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30f40098a0 tx=0x7f30f4009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.994+0000 7f3109800700 1 -- 172.21.15.7:0/528959967 shutdown_connections 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.994+0000 7f3109800700 1 --2- 172.21.15.7:0/528959967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f3104106b50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.994+0000 7f3109800700 1 -- 172.21.15.7:0/528959967 >> 172.21.15.7:0/528959967 conn(0x7f31041020a0 msgr2=0x7f31041044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.994+0000 7f3109800700 1 -- 172.21.15.7:0/528959967 shutdown_connections 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.994+0000 7f3109800700 1 -- 172.21.15.7:0/528959967 wait complete. 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.995+0000 7f3109800700 1 Processor -- start 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.995+0000 7f3109800700 1 -- start start 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.995+0000 7f3109800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.995+0000 7f3109800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f310410ca70 con 0x7f3104106730 2024-04-20T21:21:49.748 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.996+0000 7f3103e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.996+0000 7f3103e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34022/0 (socket says 172.21.15.7:34022) 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.996+0000 7f3103e00700 1 -- 172.21.15.7:0/1583245038 learned_addr learned my addr 172.21.15.7:0/1583245038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.996+0000 7f3103e00700 1 -- 172.21.15.7:0/1583245038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310419c900 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.996+0000 7f3103e00700 1 --2- 172.21.15.7:0/1583245038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f30f4011850 tx=0x7f30f4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f4003af0 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f30f4020070 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30f40195a0 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310419cb90 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f310419f880 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f30f4019be0 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.997+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f30f4011e00 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:48.998+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31040640b0 con 0x7f3104106730 2024-04-20T21:21:49.749 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.001+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f30f4003c90 con 0x7f3104106730 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.080+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f310419fa50 con 0x7f3104106730 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.083+0000 7f3100c00700 1 -- 172.21.15.7:0/1583245038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f30f4025460 con 0x7f3104106730 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 msgr2=0x7f310419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 --2- 172.21.15.7:0/1583245038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f30f4011850 tx=0x7f30f4011c10 comp rx=0 tx=0).stop 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 shutdown_connections 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 --2- 172.21.15.7:0/1583245038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104106730 0x7f310419c3c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 >> 172.21.15.7:0/1583245038 conn(0x7f31041020a0 msgr2=0x7f3104102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.084+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 shutdown_connections 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:49.085+0000 7f3109800700 1 -- 172.21.15.7:0/1583245038 wait complete. 2024-04-20T21:21:49.750 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-20T21:21:49.776 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:21:49.776 INFO:teuthology.orchestra.run.smithi007.stdout:Creating mgr... 2024-04-20T21:21:49.777 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-20T21:21:49.778 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-20T21:21:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x 2024-04-20T21:21:50.219 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-20T21:21:50.490 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968.target.wants/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@.service. 2024-04-20T21:21:50.521 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:50 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1583245038' entity='client.admin' 2024-04-20T21:21:50.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:50 smithi007 systemd[1]: Starting Ceph mgr.x for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:21:51.154 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:50 smithi007 podman[17090]: 2024-04-20 21:21:50.792769363 +0000 UTC m=+0.036865711 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:21:51.154 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:50 smithi007 podman[17090]: 2024-04-20 21:21:50.915865313 +0000 UTC m=+0.159961668 container create 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-20T21:21:51.154 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 podman[17090]: 2024-04-20 21:21:51.15380547 +0000 UTC m=+0.397901843 container init 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:21:51.293 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:21:51.293 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-04-20T21:21:51.312 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:21:51.312 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:21:51.312 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-04-20T21:21:51.312 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to enable service . firewalld.service is not available 2024-04-20T21:21:51.336 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:21:51.336 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-04-20T21:21:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:21:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:21:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-04-20T21:21:51.354 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-20T21:21:51.355 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr to start... 2024-04-20T21:21:51.355 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr... 2024-04-20T21:21:51.413 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 podman[17090]: 2024-04-20 21:21:51.164380773 +0000 UTC m=+0.408477116 container start 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T21:21:51.413 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 bash[17090]: 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 2024-04-20T21:21:51.413 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 systemd[1]: Started Ceph mgr.x for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:21:51.677 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:51.413+0000 7fc2d69ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:21:51.957 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:51.691+0000 7fc2d69ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:21:51.957 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:51 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:51.840+0000 7fc2d69ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:21:52.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:52 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1510421202' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:21:52.562 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:52 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:52.297+0000 7fc2d69ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:21:52.562 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:52 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:52.420+0000 7fc2d69ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:21:52.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:52.856 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:21:52.857 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:21:52.858 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:21:52.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:21:52.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:21:52.868 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:52.869 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.870 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:21:41.203600+0000", 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.957+0000 7fe2ef400700 1 Processor -- start 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.957+0000 7fe2ef400700 1 -- start start 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.958+0000 7fe2ef400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.958+0000 7fe2ef400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8109040 con 0x7fe2e8108650 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.959+0000 7fe2eca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.959+0000 7fe2eca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34040/0 (socket says 172.21.15.7:34040) 2024-04-20T21:21:52.871 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.959+0000 7fe2eca00700 1 -- 172.21.15.7:0/1666829522 learned_addr learned my addr 172.21.15.7:0/1666829522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.959+0000 7fe2eca00700 1 -- 172.21.15.7:0/1666829522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e8109800 con 0x7fe2e8108650 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.960+0000 7fe2eca00700 1 --2- 172.21.15.7:0/1666829522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0098a0 tx=0x7fe2dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cee77b5a45e1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.960+0000 7fe2e7400700 1 -- 172.21.15.7:0/1666829522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc004160 con 0x7fe2e8108650 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.960+0000 7fe2e7400700 1 -- 172.21.15.7:0/1666829522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2dc004300 con 0x7fe2e8108650 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.960+0000 7fe2e7400700 1 -- 172.21.15.7:0/1666829522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc011b50 con 0x7fe2e8108650 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 -- 172.21.15.7:0/1666829522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 msgr2=0x7fe2e8108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 --2- 172.21.15.7:0/1666829522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0098a0 tx=0x7fe2dc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 -- 172.21.15.7:0/1666829522 shutdown_connections 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 --2- 172.21.15.7:0/1666829522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e8108a70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 -- 172.21.15.7:0/1666829522 >> 172.21.15.7:0/1666829522 conn(0x7fe2e8103c10 msgr2=0x7fe2e8106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 -- 172.21.15.7:0/1666829522 shutdown_connections 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.961+0000 7fe2ef400700 1 -- 172.21.15.7:0/1666829522 wait complete. 2024-04-20T21:21:52.872 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.962+0000 7fe2ef400700 1 Processor -- start 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2ef400700 1 -- start start 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2ef400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2ef400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e810cbb0 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2eca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2eca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34046/0 (socket says 172.21.15.7:34046) 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.963+0000 7fe2eca00700 1 -- 172.21.15.7:0/1510421202 learned_addr learned my addr 172.21.15.7:0/1510421202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2eca00700 1 -- 172.21.15.7:0/1510421202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e819eda0 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2eca00700 1 --2- 172.21.15.7:0/1510421202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000c00 tx=0x7fe2dc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc003b20 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2dc003cc0 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc024910 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.964+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e819f030 con 0x7fe2e8108650 2024-04-20T21:21:52.873 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.965+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e81a1d20 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.965+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2dc022020 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.965+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2dc011e00 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.966+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e80640b0 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:51.969+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe2dc01acd0 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.049+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe2e81a1ef0 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.050+0000 7fe2e5600700 1 -- 172.21.15.7:0/1510421202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe2dc032030 con 0x7fe2e8108650 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 msgr2=0x7fe2e819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 --2- 172.21.15.7:0/1510421202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc000c00 tx=0x7fe2dc011c10 comp rx=0 tx=0).stop 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 shutdown_connections 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 --2- 172.21.15.7:0/1510421202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2e8108650 0x7fe2e819e860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 >> 172.21.15.7:0/1510421202 conn(0x7fe2e8103c10 msgr2=0x7fe2e81046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.051+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 shutdown_connections 2024-04-20T21:21:52.874 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:52.052+0000 7fe2ef400700 1 -- 172.21.15.7:0/1510421202 wait complete. 2024-04-20T21:21:52.875 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (1/15)... 2024-04-20T21:21:54.528 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:54 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:54.075+0000 7fc2d69ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:21:54.528 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:54 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:54.311+0000 7fc2d69ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:21:54.528 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:54 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:54.528+0000 7fc2d69ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:21:55.195 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:54 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:54.936+0000 7fc2d69ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:21:55.195 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:55 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:55.065+0000 7fc2d69ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:21:55.488 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:55 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:55.326+0000 7fc2d69ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:21:55.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:55 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:55.596+0000 7fc2d69ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:21:55.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:55 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3157738571' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:21:56.244 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:56.245 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:21:56.246 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:21:56.247 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:21:56.247 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:21:56.247 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:21:56.247 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:21:56.247 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:21:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:21:56.256 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:21:41.203600+0000", 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.488+0000 7f173be00700 1 Processor -- start 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.488+0000 7f173be00700 1 -- start start 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.489+0000 7f173be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.489+0000 7f173be00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f173c07bed0 con 0x7f173c106cc0 2024-04-20T21:21:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.489+0000 7f173aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.489+0000 7f173aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48210/0 (socket says 172.21.15.7:48210) 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.489+0000 7f173aa00700 1 -- 172.21.15.7:0/2599222311 learned_addr learned my addr 172.21.15.7:0/2599222311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.490+0000 7f173aa00700 1 -- 172.21.15.7:0/2599222311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c109630 con 0x7f173c106cc0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.490+0000 7f173aa00700 1 --2- 172.21.15.7:0/2599222311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f172c0098a0 tx=0x7f172c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b9bd114173b017 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.490+0000 7f173a000700 1 -- 172.21.15.7:0/2599222311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c004160 con 0x7f173c106cc0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173a000700 1 -- 172.21.15.7:0/2599222311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f172c004300 con 0x7f173c106cc0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173a000700 1 -- 172.21.15.7:0/2599222311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c011b50 con 0x7f173c106cc0 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173be00700 1 -- 172.21.15.7:0/2599222311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 msgr2=0x7f173c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173be00700 1 --2- 172.21.15.7:0/2599222311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f172c0098a0 tx=0x7f172c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173be00700 1 -- 172.21.15.7:0/2599222311 shutdown_connections 2024-04-20T21:21:56.258 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173be00700 1 --2- 172.21.15.7:0/2599222311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1090f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.491+0000 7f173be00700 1 -- 172.21.15.7:0/2599222311 >> 172.21.15.7:0/2599222311 conn(0x7f173c102920 msgr2=0x7f173c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.492+0000 7f173be00700 1 -- 172.21.15.7:0/2599222311 shutdown_connections 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.492+0000 7f173be00700 1 -- 172.21.15.7:0/2599222311 wait complete. 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.493+0000 7f173be00700 1 Processor -- start 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.493+0000 7f173be00700 1 -- start start 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.493+0000 7f173be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.493+0000 7f173be00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f173c10a6c0 con 0x7f173c106cc0 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.494+0000 7f173aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.494+0000 7f173aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48226/0 (socket says 172.21.15.7:48226) 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.494+0000 7f173aa00700 1 -- 172.21.15.7:0/3157738571 learned_addr learned my addr 172.21.15.7:0/3157738571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.494+0000 7f173aa00700 1 -- 172.21.15.7:0/3157738571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c1a3010 con 0x7f173c106cc0 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.494+0000 7f173aa00700 1 --2- 172.21.15.7:0/3157738571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f172c011b60 tx=0x7f172c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:56.259 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c003970 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f172c003b10 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f172c024610 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c1a32a0 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c1a5f90 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f172c024c50 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.495+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f172c011e00 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.496+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173c0640b0 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.499+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f172c01a9f0 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.579+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f173c1a6540 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.582+0000 7f1733e00700 1 -- 172.21.15.7:0/3157738571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f172c01a430 con 0x7f173c106cc0 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 msgr2=0x7f173c1a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 --2- 172.21.15.7:0/3157738571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f172c011b60 tx=0x7f172c011c40 comp rx=0 tx=0).stop 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 shutdown_connections 2024-04-20T21:21:56.260 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 --2- 172.21.15.7:0/3157738571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173c106cc0 0x7f173c1a2ad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 >> 172.21.15.7:0/3157738571 conn(0x7f173c102920 msgr2=0x7f173c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.583+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 shutdown_connections 2024-04-20T21:21:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:55.584+0000 7f173be00700 1 -- 172.21.15.7:0/3157738571 wait complete. 2024-04-20T21:21:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (2/15)... 2024-04-20T21:21:56.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:56 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:56.433+0000 7fc2d69ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:21:57.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:57 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:57.007+0000 7fc2d69ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:21:57.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:57 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:57.138+0000 7fc2d69ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:21:57.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:57 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:57.412+0000 7fc2d69ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:21:58.409 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:58 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:58.103+0000 7fc2d69ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:21:58.409 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:58 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:58.231+0000 7fc2d69ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:21:58.712 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:58 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:58.582+0000 7fc2d69ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:21:58.962 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:58 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:58.769+0000 7fc2d69ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:21:58.962 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:58 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:58.902+0000 7fc2d69ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:21:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:21:59 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2776355558' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:21:59.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:59.062+0000 7fc2d69ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:21:59.568 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:21:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:21:59.444+0000 7fc2d69ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:21:59.571 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:21:59.574 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:21:59.575 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:21:59.584 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:21:59.585 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:21:41.203600+0000", 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:21:59.586 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.867+0000 7f3a6f200700 1 Processor -- start 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.867+0000 7f3a6f200700 1 -- start start 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.867+0000 7f3a6f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.867+0000 7f3a6f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6807c820 con 0x7f3a6807de30 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.868+0000 7f3a67e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.868+0000 7f3a67e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48230/0 (socket says 172.21.15.7:48230) 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.868+0000 7f3a67e00700 1 -- 172.21.15.7:0/442887584 learned_addr learned my addr 172.21.15.7:0/442887584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.868+0000 7f3a67e00700 1 -- 172.21.15.7:0/442887584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6807c9a0 con 0x7f3a6807de30 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.869+0000 7f3a67e00700 1 --2- 172.21.15.7:0/442887584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3a580098a0 tx=0x7f3a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659823fdb866d03b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.869+0000 7f3a66a00700 1 -- 172.21.15.7:0/442887584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58004160 con 0x7f3a6807de30 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.869+0000 7f3a66a00700 1 -- 172.21.15.7:0/442887584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a58004300 con 0x7f3a6807de30 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.869+0000 7f3a66a00700 1 -- 172.21.15.7:0/442887584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58011b50 con 0x7f3a6807de30 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 -- 172.21.15.7:0/442887584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 msgr2=0x7f3a6807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 --2- 172.21.15.7:0/442887584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3a580098a0 tx=0x7f3a58009bb0 comp rx=0 tx=0).stop 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 -- 172.21.15.7:0/442887584 shutdown_connections 2024-04-20T21:21:59.587 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 --2- 172.21.15.7:0/442887584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6807c2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 -- 172.21.15.7:0/442887584 >> 172.21.15.7:0/442887584 conn(0x7f3a68103c40 msgr2=0x7f3a68106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 -- 172.21.15.7:0/442887584 shutdown_connections 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.870+0000 7f3a6f200700 1 -- 172.21.15.7:0/442887584 wait complete. 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.871+0000 7f3a6f200700 1 Processor -- start 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.871+0000 7f3a6f200700 1 -- start start 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a6f200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a6f200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6810ec30 con 0x7f3a6807de30 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a67e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a67e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48238/0 (socket says 172.21.15.7:48238) 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a67e00700 1 -- 172.21.15.7:0/2776355558 learned_addr learned my addr 172.21.15.7:0/2776355558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.872+0000 7f3a67e00700 1 -- 172.21.15.7:0/2776355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6819ecb0 con 0x7f3a6807de30 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a67e00700 1 --2- 172.21.15.7:0/2776355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a58011b60 tx=0x7f3a58011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58003b20 con 0x7f3a6807de30 2024-04-20T21:21:59.588 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a58003cc0 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a58024840 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6819ef40 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.873+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a1c30 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.874+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a58022020 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.874+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a58011e00 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.874+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a680640b0 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.877+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3a58043e50 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.957+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a681a2170 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.958+0000 7f3a64c00700 1 -- 172.21.15.7:0/2776355558 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f3a58032020 con 0x7f3a6807de30 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 msgr2=0x7f3a6819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 --2- 172.21.15.7:0/2776355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a58011b60 tx=0x7f3a58011c40 comp rx=0 tx=0).stop 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 shutdown_connections 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 --2- 172.21.15.7:0/2776355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a6807de30 0x7f3a6819e770 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 >> 172.21.15.7:0/2776355558 conn(0x7f3a68103c40 msgr2=0x7f3a681047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 shutdown_connections 2024-04-20T21:21:59.589 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:21:58.959+0000 7f3a6f200700 1 -- 172.21.15.7:0/2776355558 wait complete. 2024-04-20T21:21:59.590 INFO:teuthology.orchestra.run.smithi007.stdout:mgr not available, waiting (3/15)... 2024-04-20T21:22:00.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:00.241+0000 7fc2d69ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:22:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:00.536+0000 7fc2d69ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:22:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: Activating manager daemon x 2024-04-20T21:22:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: mgrmap e2: x(active, starting, since 0.0134436s) 2024-04-20T21:22:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:22:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: Manager daemon x is now available 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:22:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:00 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' 2024-04-20T21:22:01.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:01 smithi007 ceph-mon[16867]: from='mgr.14100 172.21.15.7:0/466530045' entity='mgr.x' 2024-04-20T21:22:01.891 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:01 smithi007 ceph-mon[16867]: mgrmap e3: x(active, since 1.02258s) 2024-04-20T21:22:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:02 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3857619066' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-20T21:22:03.180 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:03.180 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:03.180 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:22:03.180 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "health": { 2024-04-20T21:22:03.181 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 0 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-20T21:22:03.182 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "a" 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-20T21:22:03.184 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-20T21:22:03.188 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-20T21:22:03.189 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "restful" 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ], 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-20T21:22:03.190 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "modified": "2024-04-20T21:21:41.203600+0000", 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout }, 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.195+0000 7f990a400700 1 Processor -- start 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.195+0000 7f990a400700 1 -- start start 2024-04-20T21:22:03.196 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.196+0000 7f990a400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.196+0000 7f990a400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f990410fa70 con 0x7f990410f080 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.196+0000 7f9903400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.196+0000 7f9903400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48316/0 (socket says 172.21.15.7:48316) 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.196+0000 7f9903400700 1 -- 172.21.15.7:0/2546777399 learned_addr learned my addr 172.21.15.7:0/2546777399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.197+0000 7f9903400700 1 -- 172.21.15.7:0/2546777399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990410fe40 con 0x7f990410f080 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.197+0000 7f9903400700 1 --2- 172.21.15.7:0/2546777399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98f40098a0 tx=0x7f98f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d701942bf8eda6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.197+0000 7f9902000700 1 -- 172.21.15.7:0/2546777399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4004160 con 0x7f990410f080 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.197+0000 7f9902000700 1 -- 172.21.15.7:0/2546777399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98f4004300 con 0x7f990410f080 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f9902000700 1 -- 172.21.15.7:0/2546777399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4011b50 con 0x7f990410f080 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 -- 172.21.15.7:0/2546777399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 msgr2=0x7f990410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 --2- 172.21.15.7:0/2546777399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98f40098a0 tx=0x7f98f4009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 -- 172.21.15.7:0/2546777399 shutdown_connections 2024-04-20T21:22:03.197 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 --2- 172.21.15.7:0/2546777399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f990410f4a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 -- 172.21.15.7:0/2546777399 >> 172.21.15.7:0/2546777399 conn(0x7f990407de30 msgr2=0x7f9904080270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 -- 172.21.15.7:0/2546777399 shutdown_connections 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.198+0000 7f990a400700 1 -- 172.21.15.7:0/2546777399 wait complete. 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.199+0000 7f990a400700 1 Processor -- start 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f990a400700 1 -- start start 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f990a400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f990a400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99041131f0 con 0x7f990410f080 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f9903400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f9903400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48328/0 (socket says 172.21.15.7:48328) 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.200+0000 7f9903400700 1 -- 172.21.15.7:0/3857619066 learned_addr learned my addr 172.21.15.7:0/3857619066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f9903400700 1 -- 172.21.15.7:0/3857619066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99041a3080 con 0x7f990410f080 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f9903400700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98f4011b60 tx=0x7f98f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:03.198 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4003970 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98f4003b10 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99041a3310 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.201+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f4024610 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.202+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a6000 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.203+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f98f401a9f0 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.203+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99040640b0 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.203+0000 7f98fbe00700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 0x7f98e803ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.203+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98f4011e00 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.203+0000 7f9902a00700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 0x7f98e803ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.204+0000 7f9902a00700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 0x7f98e803ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.210+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f98f4016260 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.532+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f99041a61d0 con 0x7f990410f080 2024-04-20T21:22:03.199 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.533+0000 7f98fbe00700 1 -- 172.21.15.7:0/3857619066 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f99041a61d0 con 0x7f990410f080 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.535+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 msgr2=0x7f98e803ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.535+0000 7f990a400700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 0x7f98e803ba10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fd0 tx=0x7f98f0006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.535+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 msgr2=0x7f99041a2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.535+0000 7f990a400700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f98f4011b60 tx=0x7f98f4011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 shutdown_connections 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f98e8039550 0x7f98e803ba10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:03.200 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 --2- 172.21.15.7:0/3857619066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990410f080 0x7f99041a2b40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:03.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 >> 172.21.15.7:0/3857619066 conn(0x7f990407de30 msgr2=0x7f990407f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:03.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 shutdown_connections 2024-04-20T21:22:03.201 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:02.536+0000 7f990a400700 1 -- 172.21.15.7:0/3857619066 wait complete. 2024-04-20T21:22:03.201 INFO:teuthology.orchestra.run.smithi007.stdout:mgr is available 2024-04-20T21:22:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:03 smithi007 ceph-mon[16867]: mgrmap e4: x(active, since 2s) 2024-04-20T21:22:04.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:04 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2270331256' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [global] 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout fsid = 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-20T21:22:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout [osd] 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-20T21:22:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.796+0000 7fa643000700 1 Processor -- start 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.797+0000 7fa643000700 1 -- start start 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.797+0000 7fa643000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.797+0000 7fa643000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c10b210 con 0x7fa63c10a820 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.798+0000 7fa63be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.798+0000 7fa63be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56390/0 (socket says 172.21.15.7:56390) 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.798+0000 7fa63be00700 1 -- 172.21.15.7:0/1201340955 learned_addr learned my addr 172.21.15.7:0/1201340955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.798+0000 7fa63be00700 1 -- 172.21.15.7:0/1201340955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c10ba20 con 0x7fa63c10a820 2024-04-20T21:22:04.850 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.799+0000 7fa63be00700 1 --2- 172.21.15.7:0/1201340955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0098a0 tx=0x7fa62c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e706c018bc43cfb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.799+0000 7fa63aa00700 1 -- 172.21.15.7:0/1201340955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa62c004160 con 0x7fa63c10a820 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.799+0000 7fa63aa00700 1 -- 172.21.15.7:0/1201340955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa62c004300 con 0x7fa63c10a820 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.799+0000 7fa63aa00700 1 -- 172.21.15.7:0/1201340955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa62c011b50 con 0x7fa63c10a820 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 -- 172.21.15.7:0/1201340955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 msgr2=0x7fa63c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 --2- 172.21.15.7:0/1201340955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0098a0 tx=0x7fa62c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 -- 172.21.15.7:0/1201340955 shutdown_connections 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 --2- 172.21.15.7:0/1201340955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c10ac40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 -- 172.21.15.7:0/1201340955 >> 172.21.15.7:0/1201340955 conn(0x7fa63c07de30 msgr2=0x7fa63c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 -- 172.21.15.7:0/1201340955 shutdown_connections 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.800+0000 7fa643000700 1 -- 172.21.15.7:0/1201340955 wait complete. 2024-04-20T21:22:04.851 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.801+0000 7fa643000700 1 Processor -- start 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.801+0000 7fa643000700 1 -- start start 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.801+0000 7fa643000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.801+0000 7fa643000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c10edd0 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.802+0000 7fa63be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.802+0000 7fa63be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56402/0 (socket says 172.21.15.7:56402) 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.802+0000 7fa63be00700 1 -- 172.21.15.7:0/2270331256 learned_addr learned my addr 172.21.15.7:0/2270331256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.802+0000 7fa63be00700 1 -- 172.21.15.7:0/2270331256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c19ecd0 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.802+0000 7fa63be00700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009fb0 tx=0x7fa62c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.803+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa62c0038b0 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.803+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa62c011e50 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.803+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa62c019550 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.803+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c19ef60 con 0x7fa63c10a820 2024-04-20T21:22:04.852 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.803+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1a1c50 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.804+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fa62c003a50 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.804+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c0640b0 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.805+0000 7fa638c00700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 0x7fa62003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.805+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa62c04d8c0 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.805+0000 7fa63b400700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 0x7fa62003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.806+0000 7fa63b400700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 0x7fa62003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa628006fd0 tx=0x7fa628006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:03.810+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa62c04d2e0 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.039+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa63c1a1e20 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.041+0000 7fa638c00700 1 -- 172.21.15.7:0/2270331256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7fa63c1a1e20 con 0x7fa63c10a820 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 msgr2=0x7fa62003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 0x7fa62003bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa628006fd0 tx=0x7fa628006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:04.853 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 msgr2=0x7fa63c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009fb0 tx=0x7fa62c011bf0 comp rx=0 tx=0).stop 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 shutdown_connections 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7fa620039620 0x7fa62003bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 --2- 172.21.15.7:0/2270331256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63c10a820 0x7fa63c19e790 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 >> 172.21.15.7:0/2270331256 conn(0x7fa63c07de30 msgr2=0x7fa63c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.044+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 shutdown_connections 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:04.045+0000 7fa643000700 1 -- 172.21.15.7:0/2270331256 wait complete. 2024-04-20T21:22:04.854 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling cephadm module... 2024-04-20T21:22:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:05 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1440426447' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-20T21:22:06.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:05 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setuser ceph since I am not root 2024-04-20T21:22:06.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:05 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setgroup ceph since I am not root 2024-04-20T21:22:06.354 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:06.092+0000 7f8b724a2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:22:06.354 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:06.353+0000 7f8b724a2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:06.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.446+0000 7f96df800700 1 Processor -- start 2024-04-20T21:22:06.499 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96df800700 1 -- start start 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96df800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96df800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810b210 con 0x7f96d810a820 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96dce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96dce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56404/0 (socket says 172.21.15.7:56404) 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.447+0000 7f96dce00700 1 -- 172.21.15.7:0/4058111919 learned_addr learned my addr 172.21.15.7:0/4058111919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.448+0000 7f96dce00700 1 -- 172.21.15.7:0/4058111919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d810ba20 con 0x7f96d810a820 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.448+0000 7f96dce00700 1 --2- 172.21.15.7:0/4058111919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098a0 tx=0x7f96cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f1363cc67b3ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96d7400700 1 -- 172.21.15.7:0/4058111919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc004160 con 0x7f96d810a820 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96d7400700 1 -- 172.21.15.7:0/4058111919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96cc004300 con 0x7f96d810a820 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96d7400700 1 -- 172.21.15.7:0/4058111919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc011b50 con 0x7f96d810a820 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96df800700 1 -- 172.21.15.7:0/4058111919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 msgr2=0x7f96d810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96df800700 1 --2- 172.21.15.7:0/4058111919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098a0 tx=0x7f96cc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96df800700 1 -- 172.21.15.7:0/4058111919 shutdown_connections 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96df800700 1 --2- 172.21.15.7:0/4058111919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d810a820 0x7f96d810ac40 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098a0 tx=0x7f96cc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.449+0000 7f96df800700 1 -- 172.21.15.7:0/4058111919 >> 172.21.15.7:0/4058111919 conn(0x7f96d807dc40 msgr2=0x7f96d81084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.450+0000 7f96df800700 1 -- 172.21.15.7:0/4058111919 shutdown_connections 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.450+0000 7f96df800700 1 -- 172.21.15.7:0/4058111919 wait complete. 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.451+0000 7f96df800700 1 Processor -- start 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.451+0000 7f96df800700 1 -- start start 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.451+0000 7f96df800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.451+0000 7f96df800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96d810edd0 con 0x7f96d807e6a0 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.452+0000 7f96dce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.452+0000 7f96dce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56412/0 (socket says 172.21.15.7:56412) 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.452+0000 7f96dce00700 1 -- 172.21.15.7:0/1440426447 learned_addr learned my addr 172.21.15.7:0/1440426447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:06.501 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.452+0000 7f96dce00700 1 -- 172.21.15.7:0/1440426447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d807f000 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.452+0000 7f96dce00700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96cc000c00 tx=0x7f96cc0119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.453+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc0038b0 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.453+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96cc003a50 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.453+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d807f290 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.453+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc01b440 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.453+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d8081df0 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.454+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f96cc01a440 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.454+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d80640b0 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.454+0000 7f96d5600700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 0x7f96c003bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.455+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96cc04dae0 con 0x7f96d807e6a0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.455+0000 7f96d7e00700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 0x7f96c003bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:06.502 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.455+0000 7f96d7e00700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 0x7f96c003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96c4006fd0 tx=0x7f96c4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.460+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f96cc01b6f0 con 0x7f96d807e6a0 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.752+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f96d81a6e00 con 0x7f96d807e6a0 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.806+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f96cc01a730 con 0x7f96d807e6a0 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.809+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 msgr2=0x7f96c003bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.809+0000 7f96df800700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 0x7f96c003bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f96c4006fd0 tx=0x7f96c4006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.809+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 msgr2=0x7f96d807eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96cc000c00 tx=0x7f96cc0119e0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96d5600700 1 -- 172.21.15.7:0/1440426447 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f96cc022290 con 0x7f96d807e6a0 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 shutdown_connections 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f96c0039670 0x7f96c003bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 --2- 172.21.15.7:0/1440426447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96d807e6a0 0x7f96d807eac0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 >> 172.21.15.7:0/1440426447 conn(0x7f96d807dc40 msgr2=0x7f96d8107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.811+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 shutdown_connections 2024-04-20T21:22:06.504 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:05.812+0000 7f96df800700 1 -- 172.21.15.7:0/1440426447 wait complete. 2024-04-20T21:22:06.655 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:06.489+0000 7f8b724a2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:22:06.907 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1440426447' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-20T21:22:06.907 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-mon[16867]: mgrmap e5: x(active, since 5s) 2024-04-20T21:22:07.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:06.909+0000 7f8b724a2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:22:07.269 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:07 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:07.027+0000 7f8b724a2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:22:08.012 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2305607362' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.060+0000 7f6e74400700 1 Processor -- start 2024-04-20T21:22:08.024 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.060+0000 7f6e74400700 1 -- start start 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.061+0000 7f6e74400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.061+0000 7f6e74400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c10ba20 con 0x7f6e6c1090f0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.061+0000 7f6e71a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.061+0000 7f6e71a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56432/0 (socket says 172.21.15.7:56432) 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.061+0000 7f6e71a00700 1 -- 172.21.15.7:0/1714736258 learned_addr learned my addr 172.21.15.7:0/1714736258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.062+0000 7f6e71a00700 1 -- 172.21.15.7:0/1714736258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c10bba0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.062+0000 7f6e71a00700 1 --2- 172.21.15.7:0/1714736258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e600098a0 tx=0x7f6e60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc752dcaf050ec2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.062+0000 7f6e6be00700 1 -- 172.21.15.7:0/1714736258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e60004160 con 0x7f6e6c1090f0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.062+0000 7f6e6be00700 1 -- 172.21.15.7:0/1714736258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e60004300 con 0x7f6e6c1090f0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.062+0000 7f6e6be00700 1 -- 172.21.15.7:0/1714736258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e60011b50 con 0x7f6e6c1090f0 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 -- 172.21.15.7:0/1714736258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 msgr2=0x7f6e6c10b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 --2- 172.21.15.7:0/1714736258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e600098a0 tx=0x7f6e60009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 -- 172.21.15.7:0/1714736258 shutdown_connections 2024-04-20T21:22:08.025 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 --2- 172.21.15.7:0/1714736258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c10b4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 -- 172.21.15.7:0/1714736258 >> 172.21.15.7:0/1714736258 conn(0x7f6e6c102b00 msgr2=0x7f6e6c104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 -- 172.21.15.7:0/1714736258 shutdown_connections 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.063+0000 7f6e74400700 1 -- 172.21.15.7:0/1714736258 wait complete. 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.064+0000 7f6e74400700 1 Processor -- start 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e74400700 1 -- start start 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e74400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e74400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c10cb10 con 0x7f6e6c1090f0 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e71a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e71a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56444/0 (socket says 172.21.15.7:56444) 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.065+0000 7f6e71a00700 1 -- 172.21.15.7:0/2305607362 learned_addr learned my addr 172.21.15.7:0/2305607362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e71a00700 1 -- 172.21.15.7:0/2305607362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c19cc50 con 0x7f6e6c1090f0 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e71a00700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e60011b60 tx=0x7f6e60011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:08.026 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e60003970 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e60003b10 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e60024610 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c19cee0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.066+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c19fbd0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.067+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c0640b0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.068+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6e600248c0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.068+0000 7f6e6a000700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 0x7f6e5403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.068+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e60011e00 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.068+0000 7f6e71000700 1 -- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 msgr2=0x7f6e5403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.068+0000 7f6e71000700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 0x7f6e5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.073+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6e6000b3e0 con 0x7f6e6c1090f0 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.269+0000 7f6e71000700 1 -- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 msgr2=0x7f6e5403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:08.027 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.269+0000 7f6e71000700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 0x7f6e5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.358+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6e6c1a0110 con 0x7f6e6c1090f0 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.359+0000 7f6e6a000700 1 -- 172.21.15.7:0/2305607362 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6e60016260 con 0x7f6e6c1090f0 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 msgr2=0x7f6e5403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 0x7f6e5403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 msgr2=0x7f6e6c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e60011b60 tx=0x7f6e60011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 shutdown_connections 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f6e54039600 0x7f6e5403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 --2- 172.21.15.7:0/2305607362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e6c1090f0 0x7f6e6c19c710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 >> 172.21.15.7:0/2305607362 conn(0x7f6e6c102b00 msgr2=0x7f6e6c1036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 shutdown_connections 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:07.362+0000 7f6e74400700 1 -- 172.21.15.7:0/2305607362 wait complete. 2024-04-20T21:22:08.028 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-04-20T21:22:08.029 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 5... 2024-04-20T21:22:09.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:08 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:08.639+0000 7f8b724a2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:22:09.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:08 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:08.874+0000 7f8b724a2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:09.486 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:09.093+0000 7f8b724a2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:22:09.486 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:09.485+0000 7f8b724a2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:22:09.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:09.611+0000 7f8b724a2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:22:10.132 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:09.869+0000 7f8b724a2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:22:10.133 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:10 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:10.132+0000 7f8b724a2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:11.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:10 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:10.944+0000 7f8b724a2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:22:11.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:11.511+0000 7f8b724a2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:22:11.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:11.641+0000 7f8b724a2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:22:12.282 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:11.917+0000 7f8b724a2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:22:13.032 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:12.582+0000 7f8b724a2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:22:13.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:12.711+0000 7f8b724a2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:22:13.354 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:13 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:13.041+0000 7f8b724a2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:22:13.354 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:13 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:13.224+0000 7f8b724a2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:22:13.354 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:13 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:13.353+0000 7f8b724a2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:22:13.783 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:13 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:13.508+0000 7f8b724a2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:14.282 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:13 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:13.882+0000 7f8b724a2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:22:14.960 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:14 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:14.666+0000 7f8b724a2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:14 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:14.959+0000 7f8b724a2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: Active manager daemon x restarted 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: Activating manager daemon x 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: osdmap e2: 0 total, 0 up, 0 in 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: mgrmap e6: x(active, starting, since 0.00712081s) 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:22:15.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:22:15.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:22:15.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:22:15.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: Manager daemon x is now available 2024-04-20T21:22:15.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:15.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:15 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: Found migration_current of "None". Setting to last migration. 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:22:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:16 smithi007 ceph-mon[16867]: mgrmap e7: x(active, since 1.01718s) 2024-04-20T21:22:16.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:16.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-20T21:22:16.648 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.649+0000 7f5dc8000700 1 Processor -- start 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.649+0000 7f5dc8000700 1 -- start start 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.649+0000 7f5dc8000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.650+0000 7f5dc8000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc007c820 con 0x7f5dc007de30 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.650+0000 7f5dc5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.650+0000 7f5dc5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56456/0 (socket says 172.21.15.7:56456) 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.650+0000 7f5dc5600700 1 -- 172.21.15.7:0/1890223815 learned_addr learned my addr 172.21.15.7:0/1890223815 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:16.649 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.650+0000 7f5dc5600700 1 -- 172.21.15.7:0/1890223815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc007c9a0 con 0x7f5dc007de30 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.651+0000 7f5dc5600700 1 --2- 172.21.15.7:0/1890223815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e80 tx=0x7f5db400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac68121917637c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.651+0000 7f5dbfe00700 1 -- 172.21.15.7:0/1890223815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db4017070 con 0x7f5dc007de30 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.651+0000 7f5dbfe00700 1 -- 172.21.15.7:0/1890223815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5db400b8c0 con 0x7f5dc007de30 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.651+0000 7f5dbfe00700 1 -- 172.21.15.7:0/1890223815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db40045c0 con 0x7f5dc007de30 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 -- 172.21.15.7:0/1890223815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 msgr2=0x7f5dc007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:16.650 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 --2- 172.21.15.7:0/1890223815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009e80 tx=0x7f5db400b330 comp rx=0 tx=0).stop 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 -- 172.21.15.7:0/1890223815 shutdown_connections 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 --2- 172.21.15.7:0/1890223815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc007c2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 -- 172.21.15.7:0/1890223815 >> 172.21.15.7:0/1890223815 conn(0x7f5dc0103c10 msgr2=0x7f5dc0106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 -- 172.21.15.7:0/1890223815 shutdown_connections 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.652+0000 7f5dc8000700 1 -- 172.21.15.7:0/1890223815 wait complete. 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.653+0000 7f5dc8000700 1 Processor -- start 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc8000700 1 -- start start 2024-04-20T21:22:16.658 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc8000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc8000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc010ec40 con 0x7f5dc007de30 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56462/0 (socket says 172.21.15.7:56462) 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.654+0000 7f5dc5600700 1 -- 172.21.15.7:0/3145124295 learned_addr learned my addr 172.21.15.7:0/3145124295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dc5600700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc019eca0 con 0x7f5dc007de30 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dc5600700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5db4004350 tx=0x7f5db400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db4020020 con 0x7f5dc007de30 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5db4003c10 con 0x7f5dc007de30 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db4017040 con 0x7f5dc007de30 2024-04-20T21:22:16.659 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc019ef30 con 0x7f5dc007de30 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.655+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a1c20 con 0x7f5dc007de30 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5db4023020 con 0x7f5dc007de30 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dbe000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5da803c1d0 con 0x7f5da8039600 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5db400f050 con 0x7f5dc007de30 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.657+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.858+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:08.858+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:09.259+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.660 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:09.259+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:10.060+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:10.060+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:11.662+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:11.662+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:14.865+0000 7f5dc4c00700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2055239987 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:14.865+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:14.968+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f5db4012a10 con 0x7f5dc007de30 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:14.968+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 msgr2=0x7f5da803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:14.968+0000 7f5dbe000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.973+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5db404f1a0 con 0x7f5dc007de30 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.974+0000 7f5dbe000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 0x7f5da803f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.974+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5db40020b0 con 0x7f5da803cef0 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.975+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 0x7f5da803f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.977+0000 7f5dc4c00700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 0x7f5da803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5dac003a10 tx=0x7f5dac0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:16.661 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.979+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f5db40020b0 con 0x7f5da803cef0 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.991+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5dc00640b0 con 0x7f5da803cef0 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.992+0000 7f5dbe000700 1 -- 172.21.15.7:0/3145124295 <== mgr.14120 v2:172.21.15.7:6800/2119469525 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5dc00640b0 con 0x7f5da803cef0 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 msgr2=0x7f5da803f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 0x7f5da803f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5dac003a10 tx=0x7f5dac0092b0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 msgr2=0x7f5dc019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5db4004350 tx=0x7f5db400bd50 comp rx=0 tx=0).stop 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 shutdown_connections 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f5da803cef0 0x7f5da803f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc007de30 0x7f5dc019e760 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 --2- 172.21.15.7:0/3145124295 >> [v2:172.21.15.7:6800/2055239987,v1:172.21.15.7:6801/2055239987] conn(0x7f5da8039600 0x7f5da803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 >> 172.21.15.7:0/3145124295 conn(0x7f5dc0103c10 msgr2=0x7f5dc0104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 shutdown_connections 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:15.993+0000 7f5dc8000700 1 -- 172.21.15.7:0/3145124295 wait complete. 2024-04-20T21:22:16.662 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 5 is available 2024-04-20T21:22:16.663 INFO:teuthology.orchestra.run.smithi007.stdout:Setting orchestrator backend to cephadm... 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:16] ENGINE Bus STARTING 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:16] ENGINE Serving on http://172.21.15.7:8765 2024-04-20T21:22:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:17 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:18.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:16] ENGINE Serving on https://172.21.15.7:7150 2024-04-20T21:22:18.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:16] ENGINE Bus STARTED 2024-04-20T21:22:18.250 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: mgrmap e8: x(active, since 2s) 2024-04-20T21:22:18.250 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:18.250 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:18.250 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:18 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:18.262 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.304+0000 7f6022000700 1 Processor -- start 2024-04-20T21:22:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.305+0000 7f6022000700 1 -- start start 2024-04-20T21:22:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.305+0000 7f6022000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.305+0000 7f6022000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f601c10b080 con 0x7f601c10a690 2024-04-20T21:22:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.306+0000 7f6020c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.266 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.306+0000 7f6020c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50450/0 (socket says 172.21.15.7:50450) 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.306+0000 7f6020c00700 1 -- 172.21.15.7:0/2846328204 learned_addr learned my addr 172.21.15.7:0/2846328204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.306+0000 7f6020c00700 1 -- 172.21.15.7:0/2846328204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c10b840 con 0x7f601c10a690 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.306+0000 7f6020c00700 1 --2- 172.21.15.7:0/2846328204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60100098a0 tx=0x7f6010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50cab54d159d311 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f601b400700 1 -- 172.21.15.7:0/2846328204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6010004160 con 0x7f601c10a690 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f601b400700 1 -- 172.21.15.7:0/2846328204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6010004300 con 0x7f601c10a690 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f601b400700 1 -- 172.21.15.7:0/2846328204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6010011b50 con 0x7f601c10a690 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 -- 172.21.15.7:0/2846328204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 msgr2=0x7f601c10aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 --2- 172.21.15.7:0/2846328204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f60100098a0 tx=0x7f6010009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 -- 172.21.15.7:0/2846328204 shutdown_connections 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 --2- 172.21.15.7:0/2846328204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c10aab0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 -- 172.21.15.7:0/2846328204 >> 172.21.15.7:0/2846328204 conn(0x7f601c07dc10 msgr2=0x7f601c1084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:18.267 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 -- 172.21.15.7:0/2846328204 shutdown_connections 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.307+0000 7f6022000700 1 -- 172.21.15.7:0/2846328204 wait complete. 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.308+0000 7f6022000700 1 Processor -- start 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6022000700 1 -- start start 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6022000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6022000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f601c10ebf0 con 0x7f601c10a690 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6020c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6020c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50456/0 (socket says 172.21.15.7:50456) 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.309+0000 7f6020c00700 1 -- 172.21.15.7:0/1361094411 learned_addr learned my addr 172.21.15.7:0/1361094411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6020c00700 1 -- 172.21.15.7:0/1361094411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c19eaf0 con 0x7f601c10a690 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6020c00700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6010009fb0 tx=0x7f6010011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.268 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60100038b0 con 0x7f601c10a690 2024-04-20T21:22:18.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6010011e50 con 0x7f601c10a690 2024-04-20T21:22:18.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c19ed80 con 0x7f601c10a690 2024-04-20T21:22:18.271 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6010019550 con 0x7f601c10a690 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.310+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c1a1a70 con 0x7f601c10a690 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.311+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f601001a430 con 0x7f601c10a690 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.311+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c0640b0 con 0x7f601c10a690 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.312+0000 7f6019600700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 0x7f600403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.312+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f601004d9d0 con 0x7f601c10a690 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.312+0000 7f601be00700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 0x7f600403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.313+0000 7f601be00700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 0x7f600403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.318+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6010020240 con 0x7f601c10a690 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.566+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f601c108320 con 0x7f6004039670 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.582+0000 7f6019600700 1 -- 172.21.15.7:0/1361094411 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f601c108320 con 0x7f6004039670 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 msgr2=0x7f600403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 0x7f600403bb30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6008006fd0 tx=0x7f6008006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 msgr2=0x7f601c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6010009fb0 tx=0x7f6010011bf0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 shutdown_connections 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f6004039670 0x7f600403bb30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 --2- 172.21.15.7:0/1361094411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f601c10a690 0x7f601c19e5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 >> 172.21.15.7:0/1361094411 conn(0x7f601c07dc10 msgr2=0x7f601c107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 shutdown_connections 2024-04-20T21:22:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:17.586+0000 7f6022000700 1 -- 172.21.15.7:0/1361094411 wait complete. 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.876+0000 7fd60b200700 1 Processor -- start 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.876+0000 7fd60b200700 1 -- start start 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd60b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd60b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c107d50 con 0x7fd60c107360 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd609e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:19.827 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd609e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50470/0 (socket says 172.21.15.7:50470) 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd609e00700 1 -- 172.21.15.7:0/779889136 learned_addr learned my addr 172.21.15.7:0/779889136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd609e00700 1 -- 172.21.15.7:0/779889136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c108560 con 0x7fd60c107360 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.877+0000 7fd609e00700 1 --2- 172.21.15.7:0/779889136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0098a0 tx=0x7fd5fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628eb9728628dff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.878+0000 7fd608a00700 1 -- 172.21.15.7:0/779889136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5fc004160 con 0x7fd60c107360 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.878+0000 7fd608a00700 1 -- 172.21.15.7:0/779889136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5fc004300 con 0x7fd60c107360 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.878+0000 7fd608a00700 1 -- 172.21.15.7:0/779889136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5fc011b50 con 0x7fd60c107360 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 -- 172.21.15.7:0/779889136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 msgr2=0x7fd60c107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 --2- 172.21.15.7:0/779889136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0098a0 tx=0x7fd5fc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 -- 172.21.15.7:0/779889136 shutdown_connections 2024-04-20T21:22:19.828 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 --2- 172.21.15.7:0/779889136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c107780 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 -- 172.21.15.7:0/779889136 >> 172.21.15.7:0/779889136 conn(0x7fd60c102920 msgr2=0x7fd60c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 -- 172.21.15.7:0/779889136 shutdown_connections 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.879+0000 7fd60b200700 1 -- 172.21.15.7:0/779889136 wait complete. 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.880+0000 7fd60b200700 1 Processor -- start 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd60b200700 1 -- start start 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd60b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd60b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c10b910 con 0x7fd60c107360 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd609e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd609e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50474/0 (socket says 172.21.15.7:50474) 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.881+0000 7fd609e00700 1 -- 172.21.15.7:0/178752659 learned_addr learned my addr 172.21.15.7:0/178752659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:19.829 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd609e00700 1 -- 172.21.15.7:0/178752659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19cab0 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd609e00700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc011850 tx=0x7fd5fc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5fc003af0 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5fc020070 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19cd40 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.882+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5fc0195a0 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.883+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c19f9e0 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.884+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c0640b0 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.884+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd5fc019850 con 0x7fd60c107360 2024-04-20T21:22:19.830 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.884+0000 7fd602a00700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 0x7fd5f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.884+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5fc011e00 con 0x7fd60c107360 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.885+0000 7fd609400700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 0x7fd5f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.885+0000 7fd609400700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 0x7fd5f003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006fd0 tx=0x7fd5f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:18.891+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd5fc00e260 con 0x7fd60c107360 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.125+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd60c0008d0 con 0x7fd5f0039600 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.126+0000 7fd602a00700 1 -- 172.21.15.7:0/178752659 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd60c0008d0 con 0x7fd5f0039600 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.130+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 msgr2=0x7fd5f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.130+0000 7fd60b200700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 0x7fd5f003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006fd0 tx=0x7fd5f8006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:19.831 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.130+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 msgr2=0x7fd60c19c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.130+0000 7fd60b200700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc011850 tx=0x7fd5fc011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 shutdown_connections 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fd5f0039600 0x7fd5f003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 --2- 172.21.15.7:0/178752659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd60c107360 0x7fd60c19c570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 >> 172.21.15.7:0/178752659 conn(0x7fd60c102920 msgr2=0x7fd60c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 shutdown_connections 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:19.131+0000 7fd60b200700 1 -- 172.21.15.7:0/178752659 wait complete. 2024-04-20T21:22:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:Generating ssh key... 2024-04-20T21:22:20.111 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:19 smithi007 ceph-mon[16867]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: Generating public/private rsa key pair. 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: Your identification has been saved in /tmp/tmpdzvfzh1g/key. 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: Your public key has been saved in /tmp/tmpdzvfzh1g/key.pub. 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: The key fingerprint is: 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: SHA256:+iV2ZJhRANdTQLd3BaJNsAqcQMkpWXbXMtxgG9QTsHg ceph-8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:21.034 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: The key's randomart image is: 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: +---[RSA 3072]----+ 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | ==oo+XX*== ...| 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | o.++ *+=B* o .| 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | . = Eoooo . .| 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | o = . . | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | S o | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | . o | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | . o o | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | o + | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: | . | 2024-04-20T21:22:21.035 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: +----[SHA256]-----+ 2024-04-20T21:22:21.489 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.397+0000 7fe38b600700 1 Processor -- start 2024-04-20T21:22:21.494 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.398+0000 7fe38b600700 1 -- start start 2024-04-20T21:22:21.494 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.398+0000 7fe38b600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.494 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.398+0000 7fe38b600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe384109560 con 0x7fe384108b70 2024-04-20T21:22:21.494 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.399+0000 7fe388c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.399+0000 7fe388c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50482/0 (socket says 172.21.15.7:50482) 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.399+0000 7fe388c00700 1 -- 172.21.15.7:0/2645684127 learned_addr learned my addr 172.21.15.7:0/2645684127 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.399+0000 7fe388c00700 1 -- 172.21.15.7:0/2645684127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3841077d0 con 0x7fe384108b70 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.400+0000 7fe388c00700 1 --2- 172.21.15.7:0/2645684127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe37800bb40 tx=0x7fe37800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730f3e098f288983 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.400+0000 7fe383400700 1 -- 172.21.15.7:0/2645684127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe37800dbe0 con 0x7fe384108b70 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.400+0000 7fe383400700 1 -- 172.21.15.7:0/2645684127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3780044f0 con 0x7fe384108b70 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.400+0000 7fe383400700 1 -- 172.21.15.7:0/2645684127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe378010470 con 0x7fe384108b70 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 -- 172.21.15.7:0/2645684127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 msgr2=0x7fe384108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 --2- 172.21.15.7:0/2645684127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe37800bb40 tx=0x7fe37800be50 comp rx=0 tx=0).stop 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 -- 172.21.15.7:0/2645684127 shutdown_connections 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 --2- 172.21.15.7:0/2645684127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe384108f90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 -- 172.21.15.7:0/2645684127 >> 172.21.15.7:0/2645684127 conn(0x7fe384102300 msgr2=0x7fe384104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:21.495 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 -- 172.21.15.7:0/2645684127 shutdown_connections 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.401+0000 7fe38b600700 1 -- 172.21.15.7:0/2645684127 wait complete. 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.402+0000 7fe38b600700 1 Processor -- start 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.402+0000 7fe38b600700 1 -- start start 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe38b600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe38b600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3841066c0 con 0x7fe384108b70 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe388c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe388c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50488/0 (socket says 172.21.15.7:50488) 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe388c00700 1 -- 172.21.15.7:0/2880811070 learned_addr learned my addr 172.21.15.7:0/2880811070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.403+0000 7fe388c00700 1 -- 172.21.15.7:0/2880811070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38419a920 con 0x7fe384108b70 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe388c00700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b340 tx=0x7fe37800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe378027070 con 0x7fe384108b70 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe378003dd0 con 0x7fe384108b70 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38419abb0 con 0x7fe384108b70 2024-04-20T21:22:21.496 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe378022ac0 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.404+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38419d850 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.405+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3840640b0 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.406+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe37802a070 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.406+0000 7fe381600700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 0x7fe36c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.406+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe37801f070 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.407+0000 7fe383e00700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 0x7fe36c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.407+0000 7fe383e00700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 0x7fe36c03bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe370006fd0 tx=0x7fe370006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.411+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe378053030 con 0x7fe384108b70 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.631+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe384062e90 con 0x7fe36c0396c0 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.766+0000 7fe381600700 1 -- 172.21.15.7:0/2880811070 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe384062e90 con 0x7fe36c0396c0 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.768+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 msgr2=0x7fe36c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 0x7fe36c03bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe370006fd0 tx=0x7fe370006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 msgr2=0x7fe38419a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:21.497 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b340 tx=0x7fe37800dd90 comp rx=0 tx=0).stop 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 shutdown_connections 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fe36c0396c0 0x7fe36c03bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 --2- 172.21.15.7:0/2880811070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe384108b70 0x7fe38419a3e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 >> 172.21.15.7:0/2880811070 conn(0x7fe384102300 msgr2=0x7fe384190a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 shutdown_connections 2024-04-20T21:22:21.498 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:20.769+0000 7fe38b600700 1 -- 172.21.15.7:0/2880811070 wait complete. 2024-04-20T21:22:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:21 smithi007 ceph-mon[16867]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:21 smithi007 ceph-mon[16867]: Generating ssh key... 2024-04-20T21:22:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:21 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:21 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:22.935 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.077+0000 7fdacce00700 1 Processor -- start 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.078+0000 7fdacce00700 1 -- start start 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.078+0000 7fdacce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.078+0000 7fdacce00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8108ff0 con 0x7fdac8108600 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.079+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.079+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50492/0 (socket says 172.21.15.7:50492) 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.079+0000 7fdac5e00700 1 -- 172.21.15.7:0/611139155 learned_addr learned my addr 172.21.15.7:0/611139155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.079+0000 7fdac5e00700 1 -- 172.21.15.7:0/611139155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac8109800 con 0x7fdac8108600 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.079+0000 7fdac5e00700 1 --2- 172.21.15.7:0/611139155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098a0 tx=0x7fdab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caad11df262fa7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.080+0000 7fdac4a00700 1 -- 172.21.15.7:0/611139155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8004160 con 0x7fdac8108600 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.080+0000 7fdac4a00700 1 -- 172.21.15.7:0/611139155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdab8004300 con 0x7fdac8108600 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.080+0000 7fdac4a00700 1 -- 172.21.15.7:0/611139155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8011b50 con 0x7fdac8108600 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 -- 172.21.15.7:0/611139155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 msgr2=0x7fdac8108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 --2- 172.21.15.7:0/611139155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdab80098a0 tx=0x7fdab8009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 -- 172.21.15.7:0/611139155 shutdown_connections 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 --2- 172.21.15.7:0/611139155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac8108a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 -- 172.21.15.7:0/611139155 >> 172.21.15.7:0/611139155 conn(0x7fdac8103be0 msgr2=0x7fdac8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 -- 172.21.15.7:0/611139155 shutdown_connections 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.081+0000 7fdacce00700 1 -- 172.21.15.7:0/611139155 wait complete. 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.082+0000 7fdacce00700 1 Processor -- start 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.082+0000 7fdacce00700 1 -- start start 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.082+0000 7fdacce00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.082+0000 7fdacce00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdac810cbb0 con 0x7fdac8108600 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.083+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.083+0000 7fdac5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50506/0 (socket says 172.21.15.7:50506) 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.083+0000 7fdac5e00700 1 -- 172.21.15.7:0/1162434707 learned_addr learned my addr 172.21.15.7:0/1162434707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.083+0000 7fdac5e00700 1 -- 172.21.15.7:0/1162434707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819ecd0 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.083+0000 7fdac5e00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009fb0 tx=0x7fdab8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.084+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab80038b0 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.084+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdab8011e50 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.084+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819ef60 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.084+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdab8019550 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.084+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a1c50 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.085+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac80640b0 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.085+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdab8003a50 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.085+0000 7fdabea00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 0x7fdaac03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.085+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdab804da30 con 0x7fdac8108600 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.086+0000 7fdac5400700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 0x7fdaac03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.086+0000 7fdac5400700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 0x7fdaac03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006fd0 tx=0x7fdab4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.091+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdab804d380 con 0x7fdac8108600 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.322+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fdac8104d10 con 0x7fdaac039670 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.323+0000 7fdabea00700 1 -- 172.21.15.7:0/1162434707 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fdac8104d10 con 0x7fdaac039670 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.326+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 msgr2=0x7fdaac03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.326+0000 7fdacce00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 0x7fdaac03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdab4006fd0 tx=0x7fdab4006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.326+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 msgr2=0x7fdac819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.326+0000 7fdacce00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009fb0 tx=0x7fdab8011bf0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.326+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 shutdown_connections 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.327+0000 7fdacce00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fdaac039670 0x7fdaac03bb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.327+0000 7fdacce00700 1 --2- 172.21.15.7:0/1162434707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdac8108600 0x7fdac819e790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.327+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 >> 172.21.15.7:0/1162434707 conn(0x7fdac8103be0 msgr2=0x7fdac8104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.327+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 shutdown_connections 2024-04-20T21:22:22.940 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:22.327+0000 7fdacce00700 1 -- 172.21.15.7:0/1162434707 wait complete. 2024-04-20T21:22:22.940 INFO:teuthology.orchestra.run.smithi007.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:22:22.940 INFO:teuthology.orchestra.run.smithi007.stdout:Adding key to root@localhost authorized_keys... 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi007.stdout:Adding host smithi007... 2024-04-20T21:22:23.262 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:22 smithi007 ceph-mon[16867]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:24.225 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:23 smithi007 ceph-mon[16867]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:26.118 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:25 smithi007 ceph-mon[16867]: Deploying cephadm binary to smithi007 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Added host 'smithi007' with addr '172.21.15.7' 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.571+0000 7ff417800700 1 Processor -- start 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.572+0000 7ff417800700 1 -- start start 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.572+0000 7ff417800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.572+0000 7ff417800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff410108ff0 con 0x7ff410108600 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.572+0000 7ff414e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.573+0000 7ff414e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52966/0 (socket says 172.21.15.7:52966) 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.573+0000 7ff414e00700 1 -- 172.21.15.7:0/980736819 learned_addr learned my addr 172.21.15.7:0/980736819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.573+0000 7ff414e00700 1 -- 172.21.15.7:0/980736819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff410109800 con 0x7ff410108600 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.573+0000 7ff414e00700 1 --2- 172.21.15.7:0/980736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4040098a0 tx=0x7ff404009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c6c2807efeb4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.573+0000 7ff40f400700 1 -- 172.21.15.7:0/980736819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff404004160 con 0x7ff410108600 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff40f400700 1 -- 172.21.15.7:0/980736819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff404004300 con 0x7ff410108600 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff40f400700 1 -- 172.21.15.7:0/980736819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff404011b50 con 0x7ff410108600 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 -- 172.21.15.7:0/980736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 msgr2=0x7ff410108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 --2- 172.21.15.7:0/980736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4040098a0 tx=0x7ff404009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 -- 172.21.15.7:0/980736819 shutdown_connections 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 --2- 172.21.15.7:0/980736819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff410108a20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 -- 172.21.15.7:0/980736819 >> 172.21.15.7:0/980736819 conn(0x7ff410103be0 msgr2=0x7ff410106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.574+0000 7ff417800700 1 -- 172.21.15.7:0/980736819 shutdown_connections 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.575+0000 7ff417800700 1 -- 172.21.15.7:0/980736819 wait complete. 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff417800700 1 Processor -- start 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff417800700 1 -- start start 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff417800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff417800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff41010cbb0 con 0x7ff410108600 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff414e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff414e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52982/0 (socket says 172.21.15.7:52982) 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.576+0000 7ff414e00700 1 -- 172.21.15.7:0/1235651466 learned_addr learned my addr 172.21.15.7:0/1235651466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff414e00700 1 -- 172.21.15.7:0/1235651466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41019caa0 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff414e00700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff404011850 tx=0x7ff404011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff404003af0 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff404020070 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41019cd30 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.577+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4040195a0 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.578+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41019fa20 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.579+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4100640b0 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.579+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff404019850 con 0x7ff410108600 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.580+0000 7ff40d600700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 0x7ff3f803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.580+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff404011e00 con 0x7ff410108600 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.580+0000 7ff40fe00700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 0x7ff3f803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.581+0000 7ff40fe00700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 0x7ff3f803bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006fd0 tx=0x7ff3fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.586+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff40402a070 con 0x7ff410108600 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:23.820+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi007", "addr": "172.21.15.7", "target": ["mon-mgr", ""]}) -- 0x7ff410104dd0 con 0x7ff3f80396c0 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.092+0000 7ff40d600700 1 -- 172.21.15.7:0/1235651466 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7ff410104dd0 con 0x7ff3f80396c0 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 msgr2=0x7ff3f803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 0x7ff3f803bb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006fd0 tx=0x7ff3fc006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 msgr2=0x7ff41019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff404011850 tx=0x7ff404011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 shutdown_connections 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7ff3f80396c0 0x7ff3f803bb80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 --2- 172.21.15.7:0/1235651466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff410108600 0x7ff41019c560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.097+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 >> 172.21.15.7:0/1235651466 conn(0x7ff410103be0 msgr2=0x7ff4101046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.098+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 shutdown_connections 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:30.098+0000 7ff417800700 1 -- 172.21.15.7:0/1235651466 wait complete. 2024-04-20T21:22:30.803 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mon service... 2024-04-20T21:22:31.344 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:31 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:31.344 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:31 smithi007 ceph-mon[16867]: Added host smithi007 2024-04-20T21:22:31.344 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:31 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:32.906 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-20T21:22:32.912 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.508+0000 7f53f1600700 1 Processor -- start 2024-04-20T21:22:32.912 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.508+0000 7f53f1600700 1 -- start start 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.509+0000 7f53f1600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.509+0000 7f53f1600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec07bf70 con 0x7f53ec109110 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.509+0000 7f53eaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.510+0000 7f53eaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52984/0 (socket says 172.21.15.7:52984) 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.510+0000 7f53eaa00700 1 -- 172.21.15.7:0/943434493 learned_addr learned my addr 172.21.15.7:0/943434493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.510+0000 7f53eaa00700 1 -- 172.21.15.7:0/943434493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec10ba40 con 0x7f53ec109110 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.510+0000 7f53eaa00700 1 --2- 172.21.15.7:0/943434493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0098a0 tx=0x7f53dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7e12ec3e20f03b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.511+0000 7f53e9600700 1 -- 172.21.15.7:0/943434493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53dc004160 con 0x7f53ec109110 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.511+0000 7f53e9600700 1 -- 172.21.15.7:0/943434493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53dc004300 con 0x7f53ec109110 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.511+0000 7f53e9600700 1 -- 172.21.15.7:0/943434493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53dc011b50 con 0x7f53ec109110 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 -- 172.21.15.7:0/943434493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 msgr2=0x7f53ec10b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 --2- 172.21.15.7:0/943434493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0098a0 tx=0x7f53dc009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.913 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 -- 172.21.15.7:0/943434493 shutdown_connections 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 --2- 172.21.15.7:0/943434493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec10b500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 -- 172.21.15.7:0/943434493 >> 172.21.15.7:0/943434493 conn(0x7f53ec102b00 msgr2=0x7f53ec104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 -- 172.21.15.7:0/943434493 shutdown_connections 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.512+0000 7f53f1600700 1 -- 172.21.15.7:0/943434493 wait complete. 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.513+0000 7f53f1600700 1 Processor -- start 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.513+0000 7f53f1600700 1 -- start start 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.513+0000 7f53f1600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.513+0000 7f53f1600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec10cb20 con 0x7f53ec109110 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.514+0000 7f53eaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.514+0000 7f53eaa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52986/0 (socket says 172.21.15.7:52986) 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.514+0000 7f53eaa00700 1 -- 172.21.15.7:0/1640539135 learned_addr learned my addr 172.21.15.7:0/1640539135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.514+0000 7f53eaa00700 1 -- 172.21.15.7:0/1640539135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec108ed0 con 0x7f53ec109110 2024-04-20T21:22:32.914 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.514+0000 7f53eaa00700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53dc011b60 tx=0x7f53dc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.515+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53dc003970 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.515+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f53dc003b10 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.515+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53dc024610 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.515+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1070c0 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.515+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec107600 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.516+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f53dc01a9f0 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.516+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec0640b0 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.517+0000 7f53e3400700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 0x7f53d003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.517+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f53dc011e00 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.517+0000 7f53ea000700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 0x7f53d003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.518+0000 7f53ea000700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 0x7f53d003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.523+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f53dc019dd0 con 0x7f53ec109110 2024-04-20T21:22:32.915 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.780+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f53ec107d50 con 0x7f53d00395b0 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.790+0000 7f53e3400700 1 -- 172.21.15.7:0/1640539135 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f53ec107d50 con 0x7f53d00395b0 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 msgr2=0x7f53d003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 0x7f53d003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 msgr2=0x7f53ec108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53dc011b60 tx=0x7f53dc011c40 comp rx=0 tx=0).stop 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 shutdown_connections 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f53d00395b0 0x7f53d003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 --2- 172.21.15.7:0/1640539135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53ec109110 0x7f53ec108990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.793+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 >> 172.21.15.7:0/1640539135 conn(0x7f53ec102b00 msgr2=0x7f53ec103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.794+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 shutdown_connections 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:31.794+0000 7f53f1600700 1 -- 172.21.15.7:0/1640539135 wait complete. 2024-04-20T21:22:32.916 INFO:teuthology.orchestra.run.smithi007.stdout:Deploying unmanaged mgr service... 2024-04-20T21:22:33.064 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:32 smithi007 ceph-mon[16867]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:33.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:32 smithi007 ceph-mon[16867]: Saving service mon spec with placement count:5 2024-04-20T21:22:33.065 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:32 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.584+0000 7fbc26600700 1 Processor -- start 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.584+0000 7fbc26600700 1 -- start start 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.585+0000 7fbc26600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.585+0000 7fbc26600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2007c820 con 0x7fbc2007de30 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.586+0000 7fbc25200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.586+0000 7fbc25200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57642/0 (socket says 172.21.15.7:57642) 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.586+0000 7fbc25200700 1 -- 172.21.15.7:0/2982834219 learned_addr learned my addr 172.21.15.7:0/2982834219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.586+0000 7fbc25200700 1 -- 172.21.15.7:0/2982834219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2007c9a0 con 0x7fbc2007de30 2024-04-20T21:22:34.551 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.587+0000 7fbc25200700 1 --2- 172.21.15.7:0/2982834219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098a0 tx=0x7fbc14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec55f8424f24145 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.587+0000 7fbc1f400700 1 -- 172.21.15.7:0/2982834219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14004160 con 0x7fbc2007de30 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc1f400700 1 -- 172.21.15.7:0/2982834219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc14004300 con 0x7fbc2007de30 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc1f400700 1 -- 172.21.15.7:0/2982834219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14011b50 con 0x7fbc2007de30 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc26600700 1 -- 172.21.15.7:0/2982834219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 msgr2=0x7fbc2007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc26600700 1 --2- 172.21.15.7:0/2982834219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098a0 tx=0x7fbc14009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc26600700 1 -- 172.21.15.7:0/2982834219 shutdown_connections 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc26600700 1 --2- 172.21.15.7:0/2982834219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2007de30 0x7fbc2007c2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.588+0000 7fbc26600700 1 -- 172.21.15.7:0/2982834219 >> 172.21.15.7:0/2982834219 conn(0x7fbc20103a90 msgr2=0x7fbc20105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.589+0000 7fbc26600700 1 -- 172.21.15.7:0/2982834219 shutdown_connections 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.589+0000 7fbc26600700 1 -- 172.21.15.7:0/2982834219 wait complete. 2024-04-20T21:22:34.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.590+0000 7fbc26600700 1 Processor -- start 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.590+0000 7fbc26600700 1 -- start start 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.591+0000 7fbc26600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.591+0000 7fbc26600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2010eaa0 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.591+0000 7fbc25200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.591+0000 7fbc25200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57648/0 (socket says 172.21.15.7:57648) 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.591+0000 7fbc25200700 1 -- 172.21.15.7:0/1982465888 learned_addr learned my addr 172.21.15.7:0/1982465888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc25200700 1 -- 172.21.15.7:0/1982465888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2019f0f0 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc25200700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc140119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc140038b0 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc14003a50 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2019f380 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1401b440 con 0x7fbc2019e790 2024-04-20T21:22:34.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.592+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201a1e60 con 0x7fbc2019e790 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.593+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbc1401a440 con 0x7fbc2019e790 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.594+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc200640b0 con 0x7fbc2019e790 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.594+0000 7fbc1d600700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 0x7fbc0803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.594+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc1404db30 con 0x7fbc2019e790 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.594+0000 7fbc1fe00700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 0x7fbc0803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.595+0000 7fbc1fe00700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 0x7fbc0803bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006fd0 tx=0x7fbc0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.600+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbc1401b600 con 0x7fbc2019e790 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.848+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbc201a20f0 con 0x7fbc080396c0 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.858+0000 7fbc1d600700 1 -- 172.21.15.7:0/1982465888 <== mgr.14120 v2:172.21.15.7:6800/2119469525 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbc201a20f0 con 0x7fbc080396c0 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.861+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 msgr2=0x7fbc0803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.861+0000 7fbc26600700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 0x7fbc0803bb80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c006fd0 tx=0x7fbc0c006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:34.554 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.861+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 msgr2=0x7fbc2019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.861+0000 7fbc26600700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc140119e0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 shutdown_connections 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fbc080396c0 0x7fbc0803bb80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 --2- 172.21.15.7:0/1982465888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc2019e790 0x7fbc2019ebb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 >> 172.21.15.7:0/1982465888 conn(0x7fbc20103a90 msgr2=0x7fbc201045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 shutdown_connections 2024-04-20T21:22:34.555 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:33.862+0000 7fbc26600700 1 -- 172.21.15.7:0/1982465888 wait complete. 2024-04-20T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:34 smithi007 ceph-mon[16867]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:34 smithi007 ceph-mon[16867]: Saving service mgr spec with placement count:2 2024-04-20T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:34 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:34 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:34 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.190+0000 7f9de5400700 1 Processor -- start 2024-04-20T21:22:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.191+0000 7f9de5400700 1 -- start start 2024-04-20T21:22:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.191+0000 7f9de5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.191+0000 7f9de5400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0107d70 con 0x7f9de0107380 2024-04-20T21:22:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.191+0000 7f9ddfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.192+0000 7f9ddfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57664/0 (socket says 172.21.15.7:57664) 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.192+0000 7f9ddfe00700 1 -- 172.21.15.7:0/2583680851 learned_addr learned my addr 172.21.15.7:0/2583680851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.192+0000 7f9ddfe00700 1 -- 172.21.15.7:0/2583680851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de0108580 con 0x7f9de0107380 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.192+0000 7f9ddfe00700 1 --2- 172.21.15.7:0/2583680851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00098a0 tx=0x7f9dd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc1503031edd3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.193+0000 7f9ddea00700 1 -- 172.21.15.7:0/2583680851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0004160 con 0x7f9de0107380 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.193+0000 7f9ddea00700 1 -- 172.21.15.7:0/2583680851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd0004300 con 0x7f9de0107380 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.193+0000 7f9ddea00700 1 -- 172.21.15.7:0/2583680851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0011b20 con 0x7f9de0107380 2024-04-20T21:22:36.073 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 -- 172.21.15.7:0/2583680851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 msgr2=0x7f9de01077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 --2- 172.21.15.7:0/2583680851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00098a0 tx=0x7f9dd0009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 -- 172.21.15.7:0/2583680851 shutdown_connections 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 --2- 172.21.15.7:0/2583680851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de01077a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 -- 172.21.15.7:0/2583680851 >> 172.21.15.7:0/2583680851 conn(0x7f9de0102980 msgr2=0x7f9de0104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 -- 172.21.15.7:0/2583680851 shutdown_connections 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.194+0000 7f9de5400700 1 -- 172.21.15.7:0/2583680851 wait complete. 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.195+0000 7f9de5400700 1 Processor -- start 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.195+0000 7f9de5400700 1 -- start start 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9de5400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.078 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9de5400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9de010b930 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9ddfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9ddfe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57680/0 (socket says 172.21.15.7:57680) 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9ddfe00700 1 -- 172.21.15.7:0/2819515924 learned_addr learned my addr 172.21.15.7:0/2819515924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.196+0000 7f9ddfe00700 1 -- 172.21.15.7:0/2819515924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de0192a70 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9ddfe00700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00117e0 tx=0x7f9dd0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0003bf0 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd001a910 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0024970 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de0190c60 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.197+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de0191120 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.198+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9dd0022020 con 0x7f9de0107380 2024-04-20T21:22:36.079 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.199+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de00640b0 con 0x7f9de0107380 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.199+0000 7f9ddcc00700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 0x7f9dc403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.199+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9dd004db70 con 0x7f9de0107380 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.199+0000 7f9ddf400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 0x7f9dc403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.200+0000 7f9ddf400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 0x7f9dc403bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.204+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9dd000b3e0 con 0x7f9de0107380 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.437+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9de0196280 con 0x7f9de0107380 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.441+0000 7f9ddcc00700 1 -- 172.21.15.7:0/2819515924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f9dd002cda0 con 0x7f9de0107380 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 msgr2=0x7f9dc403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 0x7f9dc403bb80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc006fd0 tx=0x7f9dcc006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 msgr2=0x7f9de0192530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00117e0 tx=0x7f9dd0011ba0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 shutdown_connections 2024-04-20T21:22:36.080 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f9dc40396c0 0x7f9dc403bb80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.081 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 --2- 172.21.15.7:0/2819515924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de0107380 0x7f9de0192530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.081 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 >> 172.21.15.7:0/2819515924 conn(0x7f9de0102980 msgr2=0x7f9de01036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.081 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.444+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 shutdown_connections 2024-04-20T21:22:36.081 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:35.445+0000 7f9de5400700 1 -- 172.21.15.7:0/2819515924 wait complete. 2024-04-20T21:22:36.672 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:36 smithi007 ceph-mon[16867]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:22:36.672 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2819515924' entity='client.admin' 2024-04-20T21:22:37.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.673+0000 7fc3d7c00700 1 Processor -- start 2024-04-20T21:22:37.979 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.673+0000 7fc3d7c00700 1 -- start start 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.673+0000 7fc3d7c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.673+0000 7fc3d7c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0108ff0 con 0x7fc3d0108600 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.674+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.674+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57688/0 (socket says 172.21.15.7:57688) 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.674+0000 7fc3d5200700 1 -- 172.21.15.7:0/2380225548 learned_addr learned my addr 172.21.15.7:0/2380225548 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.674+0000 7fc3d5200700 1 -- 172.21.15.7:0/2380225548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d0109800 con 0x7fc3d0108600 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.675+0000 7fc3d5200700 1 --2- 172.21.15.7:0/2380225548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40098a0 tx=0x7fc3c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c500c06454ae2378 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.675+0000 7fc3cf400700 1 -- 172.21.15.7:0/2380225548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4004160 con 0x7fc3d0108600 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.675+0000 7fc3cf400700 1 -- 172.21.15.7:0/2380225548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c4004300 con 0x7fc3d0108600 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.675+0000 7fc3cf400700 1 -- 172.21.15.7:0/2380225548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4011b20 con 0x7fc3d0108600 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 -- 172.21.15.7:0/2380225548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 msgr2=0x7fc3d0108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/2380225548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40098a0 tx=0x7fc3c4009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.980 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 -- 172.21.15.7:0/2380225548 shutdown_connections 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/2380225548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d0108a20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 -- 172.21.15.7:0/2380225548 >> 172.21.15.7:0/2380225548 conn(0x7fc3d0103be0 msgr2=0x7fc3d0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 -- 172.21.15.7:0/2380225548 shutdown_connections 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.676+0000 7fc3d7c00700 1 -- 172.21.15.7:0/2380225548 wait complete. 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.677+0000 7fc3d7c00700 1 Processor -- start 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.677+0000 7fc3d7c00700 1 -- start start 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.678+0000 7fc3d7c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.678+0000 7fc3d7c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d010cbb0 con 0x7fc3d0108600 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.678+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.678+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57704/0 (socket says 172.21.15.7:57704) 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.678+0000 7fc3d5200700 1 -- 172.21.15.7:0/4072608093 learned_addr learned my addr 172.21.15.7:0/4072608093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3d5200700 1 -- 172.21.15.7:0/4072608093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d007e0d0 con 0x7fc3d0108600 2024-04-20T21:22:37.981 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3d5200700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4003b60 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3c401a910 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3c4024930 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d007c2c0 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.679+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d007c780 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.680+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc3c4022020 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.680+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d00640b0 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.681+0000 7fc3cd600700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 0x7fc3b803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.681+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc3c404d990 con 0x7fc3d0108600 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.681+0000 7fc3cfe00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 0x7fc3b803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.682+0000 7fc3cfe00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 0x7fc3b803bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0006fd0 tx=0x7fc3c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:37.982 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.686+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3c400f2e0 con 0x7fc3d0108600 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.923+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fc3d007ca00 con 0x7fc3d0108600 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.927+0000 7fc3cd600700 1 -- 172.21.15.7:0/4072608093 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fc3c404d2e0 con 0x7fc3d0108600 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.931+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 msgr2=0x7fc3b803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.931+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 0x7fc3b803bb80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0006fd0 tx=0x7fc3c0006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.931+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 msgr2=0x7fc3d007db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.931+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4000c00 tx=0x7fc3c4011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 shutdown_connections 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7fc3b80396c0 0x7fc3b803bb80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 --2- 172.21.15.7:0/4072608093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3d0108600 0x7fc3d007db90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 >> 172.21.15.7:0/4072608093 conn(0x7fc3d0103be0 msgr2=0x7fc3d0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 shutdown_connections 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:36.932+0000 7fc3d7c00700 1 -- 172.21.15.7:0/4072608093 wait complete. 2024-04-20T21:22:37.983 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling the dashboard module... 2024-04-20T21:22:38.277 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4072608093' entity='client.admin' 2024-04-20T21:22:38.277 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:37 smithi007 ceph-mon[16867]: from='mgr.14120 172.21.15.7:0/3850850240' entity='mgr.x' 2024-04-20T21:22:39.278 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:38 smithi007 ceph-mon[16867]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:22:39.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:38 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4223454949' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-20T21:22:39.279 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setuser ceph since I am not root 2024-04-20T21:22:39.279 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setgroup ceph since I am not root 2024-04-20T21:22:39.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:39.278+0000 7f51dfb25200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:22:39.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:39.532+0000 7f51dfb25200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.577+0000 7f839a200700 1 Processor -- start 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f839a200700 1 -- start start 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f839a200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f839a200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8394109000 con 0x7f8394108610 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f8393400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f8393400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57710/0 (socket says 172.21.15.7:57710) 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.578+0000 7f8393400700 1 -- 172.21.15.7:0/2934215716 learned_addr learned my addr 172.21.15.7:0/2934215716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.579+0000 7f8393400700 1 -- 172.21.15.7:0/2934215716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8394109810 con 0x7f8394108610 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.579+0000 7f8393400700 1 --2- 172.21.15.7:0/2934215716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83840098a0 tx=0x7f8384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c536ee6e1e0afab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.579+0000 7f8392a00700 1 -- 172.21.15.7:0/2934215716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8384004160 con 0x7f8394108610 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f8392a00700 1 -- 172.21.15.7:0/2934215716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8384004300 con 0x7f8394108610 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f8392a00700 1 -- 172.21.15.7:0/2934215716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8384011b20 con 0x7f8394108610 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f839a200700 1 -- 172.21.15.7:0/2934215716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 msgr2=0x7f8394108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f839a200700 1 --2- 172.21.15.7:0/2934215716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f83840098a0 tx=0x7f8384009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f839a200700 1 -- 172.21.15.7:0/2934215716 shutdown_connections 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f839a200700 1 --2- 172.21.15.7:0/2934215716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f8394108a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.580+0000 7f839a200700 1 -- 172.21.15.7:0/2934215716 >> 172.21.15.7:0/2934215716 conn(0x7f8394103cb0 msgr2=0x7f83941060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.581+0000 7f839a200700 1 -- 172.21.15.7:0/2934215716 shutdown_connections 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.581+0000 7f839a200700 1 -- 172.21.15.7:0/2934215716 wait complete. 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.582+0000 7f839a200700 1 Processor -- start 2024-04-20T21:22:39.760 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.582+0000 7f839a200700 1 -- start start 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.582+0000 7f839a200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.582+0000 7f839a200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f839410cbc0 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.583+0000 7f8393400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.583+0000 7f8393400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57722/0 (socket says 172.21.15.7:57722) 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.583+0000 7f8393400700 1 -- 172.21.15.7:0/4223454949 learned_addr learned my addr 172.21.15.7:0/4223454949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.583+0000 7f8393400700 1 -- 172.21.15.7:0/4223454949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839419c9a0 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.583+0000 7f8393400700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83840117e0 tx=0x7f8384011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.584+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8384003bf0 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.584+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838401a910 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.584+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8384024970 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.584+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839419cc30 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.584+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839419d090 con 0x7f8394108610 2024-04-20T21:22:39.761 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.585+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83940640b0 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.585+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8384022020 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.586+0000 7f8390c00700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 0x7f837803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.586+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f838404dab0 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.586+0000 7f838aa00700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 0x7f837803bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.587+0000 7f838aa00700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 0x7f837803bb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.592+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f83840212c0 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.888+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f839419fa60 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.938+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f838404d440 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.943+0000 7f8390c00700 1 -- 172.21.15.7:0/4223454949 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f838402d3f0 con 0x7f8394108610 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.943+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 msgr2=0x7f837803bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.943+0000 7f839a200700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 0x7f837803bb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.943+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 msgr2=0x7f839419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.762 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.943+0000 7f839a200700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83840117e0 tx=0x7f8384011ba0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 shutdown_connections 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f83780396c0 0x7f837803bb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 --2- 172.21.15.7:0/4223454949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8394108610 0x7f839419c460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 >> 172.21.15.7:0/4223454949 conn(0x7f8394103cb0 msgr2=0x7f8394104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 shutdown_connections 2024-04-20T21:22:39.763 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:38.945+0000 7f839a200700 1 -- 172.21.15.7:0/4223454949 wait complete. 2024-04-20T21:22:39.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:39.666+0000 7f51dfb25200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:22:40.182 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:40 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:40.080+0000 7f51dfb25200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:22:40.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4223454949' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-20T21:22:40.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:39 smithi007 ceph-mon[16867]: mgrmap e9: x(active, since 23s) 2024-04-20T21:22:40.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:40 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:40.198+0000 7f51dfb25200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:22:41.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:40 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/898604340' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-20T21:22:41.346 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:41.352 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.342+0000 7f9016600700 1 Processor -- start 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.342+0000 7f9016600700 1 -- start start 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.343+0000 7f9016600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.343+0000 7f9016600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9010109040 con 0x7f9010108650 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.343+0000 7f900f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.343+0000 7f900f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57764/0 (socket says 172.21.15.7:57764) 2024-04-20T21:22:41.353 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.343+0000 7f900f400700 1 -- 172.21.15.7:0/482207659 learned_addr learned my addr 172.21.15.7:0/482207659 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.344+0000 7f900f400700 1 -- 172.21.15.7:0/482207659 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9010109800 con 0x7f9010108650 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.344+0000 7f900f400700 1 --2- 172.21.15.7:0/482207659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90000098a0 tx=0x7f9000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9e0db5b0b6e84d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.344+0000 7f900e000700 1 -- 172.21.15.7:0/482207659 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000004160 con 0x7f9010108650 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f900e000700 1 -- 172.21.15.7:0/482207659 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9000004300 con 0x7f9010108650 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f900e000700 1 -- 172.21.15.7:0/482207659 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000011b20 con 0x7f9010108650 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 -- 172.21.15.7:0/482207659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 msgr2=0x7f9010108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 --2- 172.21.15.7:0/482207659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90000098a0 tx=0x7f9000009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 -- 172.21.15.7:0/482207659 shutdown_connections 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 --2- 172.21.15.7:0/482207659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f9010108a70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 -- 172.21.15.7:0/482207659 >> 172.21.15.7:0/482207659 conn(0x7f9010103c10 msgr2=0x7f9010106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.345+0000 7f9016600700 1 -- 172.21.15.7:0/482207659 shutdown_connections 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.346+0000 7f9016600700 1 -- 172.21.15.7:0/482207659 wait complete. 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.346+0000 7f9016600700 1 Processor -- start 2024-04-20T21:22:41.354 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f9016600700 1 -- start start 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f9016600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f9016600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f901010cbb0 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f900f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f900f400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57766/0 (socket says 172.21.15.7:57766) 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.347+0000 7f900f400700 1 -- 172.21.15.7:0/898604340 learned_addr learned my addr 172.21.15.7:0/898604340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f900f400700 1 -- 172.21.15.7:0/898604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901019cad0 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f900f400700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90000117e0 tx=0x7f9000011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9000003bf0 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f900001a910 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901019cd60 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90000249a0 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.348+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f901019d1c0 con 0x7f9010108650 2024-04-20T21:22:41.355 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.349+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f9000022020 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.350+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90100640b0 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.350+0000 7f9007e00700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 0x7f8ff403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.350+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f900004dc20 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.350+0000 7f900ea00700 1 -- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 msgr2=0x7f8ff403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.350+0000 7f900ea00700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 0x7f8ff403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.360+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9000011e20 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.551+0000 7f900ea00700 1 -- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 msgr2=0x7f8ff403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.551+0000 7f900ea00700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 0x7f8ff403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.646+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f901019fb70 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.647+0000 7f9007e00700 1 -- 172.21.15.7:0/898604340 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f900000ba00 con 0x7f9010108650 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.649+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 msgr2=0x7f8ff403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.649+0000 7f9016600700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 0x7f8ff403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.356 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.649+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 msgr2=0x7f901019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.649+0000 7f9016600700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f90000117e0 tx=0x7f9000011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 shutdown_connections 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f8ff4039610 0x7f8ff403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 --2- 172.21.15.7:0/898604340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9010108650 0x7f901019c590 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 >> 172.21.15.7:0/898604340 conn(0x7f9010103c10 msgr2=0x7f90101046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 shutdown_connections 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:40.650+0000 7f9016600700 1 -- 172.21.15.7:0/898604340 wait complete. 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for the mgr to restart... 2024-04-20T21:22:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:Waiting for mgr epoch 9... 2024-04-20T21:22:41.941 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:41 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:41.818+0000 7f51dfb25200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:22:42.282 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:42.071+0000 7f51dfb25200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:42.552 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:42.291+0000 7f51dfb25200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:22:42.809 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:42.683+0000 7f51dfb25200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:22:42.810 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:42.809+0000 7f51dfb25200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:22:43.327 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:43 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:43.067+0000 7f51dfb25200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:22:43.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:43 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:43.328+0000 7f51dfb25200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:22:44.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:44 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:44.142+0000 7f51dfb25200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:22:45.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:44 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:44.714+0000 7f51dfb25200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:22:45.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:44 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:44.844+0000 7f51dfb25200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:22:45.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:45 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:45.116+0000 7f51dfb25200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:22:46.248 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:45 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:45.787+0000 7f51dfb25200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:22:46.248 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:45 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:45.916+0000 7f51dfb25200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:22:46.248 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:46.248+0000 7f51dfb25200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:22:46.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:46.433+0000 7f51dfb25200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:22:47.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:46.565+0000 7f51dfb25200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:22:47.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:46 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:46.722+0000 7f51dfb25200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:22:47.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:47 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:47.117+0000 7f51dfb25200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:22:48.158 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:47 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:47.907+0000 7f51dfb25200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:22:48.415 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:22:48.208+0000 7f51dfb25200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: Active manager daemon x restarted 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: Activating manager daemon x 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: osdmap e3: 0 total, 0 up, 0 in 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: mgrmap e10: x(active, starting, since 0.00473828s) 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:22:48.416 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:48 smithi007 ceph-mon[16867]: Manager daemon x is now available 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:48] ENGINE Bus STARTING 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:48] ENGINE Serving on http://172.21.15.7:8765 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:48] ENGINE Serving on https://172.21.15.7:7150 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: [20/Apr/2024:21:22:48] ENGINE Bus STARTED 2024-04-20T21:22:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:49 smithi007 ceph-mon[16867]: mgrmap e11: x(active, since 1.01121s) 2024-04-20T21:22:50.387 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout { 2024-04-20T21:22:50.387 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout } 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.942+0000 7f264d000700 1 Processor -- start 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.942+0000 7f264d000700 1 -- start start 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.943+0000 7f264d000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.943+0000 7f264d000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2648109040 con 0x7f2648108650 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.943+0000 7f2645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.943+0000 7f2645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57768/0 (socket says 172.21.15.7:57768) 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.943+0000 7f2645e00700 1 -- 172.21.15.7:0/184681806 learned_addr learned my addr 172.21.15.7:0/184681806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.944+0000 7f2645e00700 1 -- 172.21.15.7:0/184681806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648109800 con 0x7f2648108650 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.944+0000 7f2645e00700 1 --2- 172.21.15.7:0/184681806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26380098a0 tx=0x7f2638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c63c1d1a1059003 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.388 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.944+0000 7f2644a00700 1 -- 172.21.15.7:0/184681806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2638004160 con 0x7f2648108650 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f2644a00700 1 -- 172.21.15.7:0/184681806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638004300 con 0x7f2648108650 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f2644a00700 1 -- 172.21.15.7:0/184681806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2638011b20 con 0x7f2648108650 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f264d000700 1 -- 172.21.15.7:0/184681806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 msgr2=0x7f2648108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f264d000700 1 --2- 172.21.15.7:0/184681806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f26380098a0 tx=0x7f2638009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f264d000700 1 -- 172.21.15.7:0/184681806 shutdown_connections 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f264d000700 1 --2- 172.21.15.7:0/184681806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f2648108a70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.945+0000 7f264d000700 1 -- 172.21.15.7:0/184681806 >> 172.21.15.7:0/184681806 conn(0x7f2648103c10 msgr2=0x7f2648106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.946+0000 7f264d000700 1 -- 172.21.15.7:0/184681806 shutdown_connections 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.946+0000 7f264d000700 1 -- 172.21.15.7:0/184681806 wait complete. 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.947+0000 7f264d000700 1 Processor -- start 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.947+0000 7f264d000700 1 -- start start 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.947+0000 7f264d000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.389 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.947+0000 7f264d000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f264810cbb0 con 0x7f2648108650 2024-04-20T21:22:50.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.948+0000 7f2645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.948+0000 7f2645e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57770/0 (socket says 172.21.15.7:57770) 2024-04-20T21:22:50.394 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.948+0000 7f2645e00700 1 -- 172.21.15.7:0/2083354213 learned_addr learned my addr 172.21.15.7:0/2083354213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.948+0000 7f2645e00700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264807e0d0 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.948+0000 7f2645e00700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26380117e0 tx=0x7f2638011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.949+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2638003bf0 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.949+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263801a910 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.949+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2638024970 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.949+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264807c2c0 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.949+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264807c780 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.950+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2638022020 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.951+0000 7f263ea00700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.951+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f262c03c230 con 0x7f262c039660 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.951+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f263804db30 con 0x7f2648108650 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.951+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:41.951+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.152+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.395 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.152+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.553+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:42.553+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.354+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:43.354+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.956+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:44.956+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:48.159+0000 7f2645400700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2119469525 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:48.159+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:48.216+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f263802d3f0 con 0x7f2648108650 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:48.216+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 msgr2=0x7f262c03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:48.216+0000 7f263ea00700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.217+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f263804fb10 con 0x7f2648108650 2024-04-20T21:22:50.396 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.218+0000 7f263ea00700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 0x7f262c03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.218+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f263804bff0 con 0x7f262c03cff0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.219+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 0x7f262c03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.220+0000 7f2645400700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 0x7f262c03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2634003a10 tx=0x7f2634009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.221+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f263804bff0 con 0x7f262c03cff0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.234+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f26480640b0 con 0x7f262c03cff0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.235+0000 7f263ea00700 1 -- 172.21.15.7:0/2083354213 <== mgr.14152 v2:172.21.15.7:6800/2375060083 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f26480640b0 con 0x7f262c03cff0 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.235+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 msgr2=0x7f262c03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.235+0000 7f264d000700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 0x7f262c03f3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2634003a10 tx=0x7f2634009270 comp rx=0 tx=0).stop 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.235+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 msgr2=0x7f264807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.235+0000 7f264d000700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26380117e0 tx=0x7f2638011ba0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 shutdown_connections 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f262c03cff0 0x7f262c03f3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2648108650 0x7f264807db90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.397 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 --2- 172.21.15.7:0/2083354213 >> [v2:172.21.15.7:6800/2119469525,v1:172.21.15.7:6801/2119469525] conn(0x7f262c039660 0x7f262c03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.398 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 >> 172.21.15.7:0/2083354213 conn(0x7f2648103c10 msgr2=0x7f26481046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.398 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 shutdown_connections 2024-04-20T21:22:50.398 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:49.236+0000 7f264d000700 1 -- 172.21.15.7:0/2083354213 wait complete. 2024-04-20T21:22:50.398 INFO:teuthology.orchestra.run.smithi007.stdout:mgr epoch 9 is available 2024-04-20T21:22:50.398 INFO:teuthology.orchestra.run.smithi007.stdout:Generating a dashboard self-signed certificate... 2024-04-20T21:22:50.530 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:50 smithi007 ceph-mon[16867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-20T21:22:50.531 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:50 smithi007 ceph-mon[16867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-20T21:22:51.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:51 smithi007 ceph-mon[16867]: mgrmap e12: x(active, since 2s) 2024-04-20T21:22:51.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:22:52.550 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.418+0000 7f0878a00700 1 Processor -- start 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.419+0000 7f0878a00700 1 -- start start 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.419+0000 7f0878a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.419+0000 7f0878a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0874108ea0 con 0x7f08741084b0 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.420+0000 7f0872800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.552 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.420+0000 7f0872800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52664/0 (socket says 172.21.15.7:52664) 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.420+0000 7f0872800700 1 -- 172.21.15.7:0/806792449 learned_addr learned my addr 172.21.15.7:0/806792449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.420+0000 7f0872800700 1 -- 172.21.15.7:0/806792449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0874109660 con 0x7f08741084b0 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.420+0000 7f0872800700 1 --2- 172.21.15.7:0/806792449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08640098a0 tx=0x7f0864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4b236a6ad92294 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.421+0000 7f0871400700 1 -- 172.21.15.7:0/806792449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0864004160 con 0x7f08741084b0 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.421+0000 7f0871400700 1 -- 172.21.15.7:0/806792449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0864004300 con 0x7f08741084b0 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.421+0000 7f0871400700 1 -- 172.21.15.7:0/806792449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0864011b20 con 0x7f08741084b0 2024-04-20T21:22:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 -- 172.21.15.7:0/806792449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 msgr2=0x7f08741088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 --2- 172.21.15.7:0/806792449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08640098a0 tx=0x7f0864009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 -- 172.21.15.7:0/806792449 shutdown_connections 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 --2- 172.21.15.7:0/806792449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f08741088d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 -- 172.21.15.7:0/806792449 >> 172.21.15.7:0/806792449 conn(0x7f0874103a90 msgr2=0x7f0874105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 -- 172.21.15.7:0/806792449 shutdown_connections 2024-04-20T21:22:52.556 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.422+0000 7f0878a00700 1 -- 172.21.15.7:0/806792449 wait complete. 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.423+0000 7f0878a00700 1 Processor -- start 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.423+0000 7f0878a00700 1 -- start start 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.424+0000 7f0878a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.424+0000 7f0878a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f087410ca10 con 0x7f08741084b0 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.424+0000 7f0872800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.424+0000 7f0872800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52678/0 (socket says 172.21.15.7:52678) 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.424+0000 7f0872800700 1 -- 172.21.15.7:0/3324939760 learned_addr learned my addr 172.21.15.7:0/3324939760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f0872800700 1 -- 172.21.15.7:0/3324939760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419c850 con 0x7f08741084b0 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f0872800700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f0864011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0864003b60 con 0x7f08741084b0 2024-04-20T21:22:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f086401a910 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0864024930 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.425+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087419cae0 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.426+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087419cf40 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.427+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740640b0 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.427+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0864022020 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.428+0000 7f086b400700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 0x7f085803bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.428+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f086404d9c0 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.428+0000 7f0871e00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 0x7f085803bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.429+0000 7f0871e00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 0x7f085803bbe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.433+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f086400f2e0 con 0x7f08741084b0 2024-04-20T21:22:52.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.670+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0874104ba0 con 0x7f0858039720 2024-04-20T21:22:52.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.861+0000 7f086b400700 1 -- 172.21.15.7:0/3324939760 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0874104ba0 con 0x7f0858039720 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 msgr2=0x7f085803bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 0x7f085803bbe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860006fd0 tx=0x7f0860006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 msgr2=0x7f087419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0864000c00 tx=0x7f0864011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 shutdown_connections 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0858039720 0x7f085803bbe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 --2- 172.21.15.7:0/3324939760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08741084b0 0x7f087419c310 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.863+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 >> 172.21.15.7:0/3324939760 conn(0x7f0874103a90 msgr2=0x7f0874104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.864+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 shutdown_connections 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:51.864+0000 7f0878a00700 1 -- 172.21.15.7:0/3324939760 wait complete. 2024-04-20T21:22:52.561 INFO:teuthology.orchestra.run.smithi007.stdout:Creating initial admin user... 2024-04-20T21:22:53.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:52 smithi007 ceph-mon[16867]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:53.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:22:53.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:22:54.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zmAInHFM81RwmySn7m6jxOigdEdt2Dh8WBgwjcL73eCnin3KxS8Me", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713648173, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-20T21:22:54.401 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.173+0000 7fac9ca00700 1 Processor -- start 2024-04-20T21:22:54.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.173+0000 7fac9ca00700 1 -- start start 2024-04-20T21:22:54.402 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.174+0000 7fac9ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.174+0000 7fac9ca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac98110a30 con 0x7fac98110040 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.175+0000 7fac97400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.175+0000 7fac97400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39948/0 (socket says 172.21.15.7:39948) 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.175+0000 7fac97400700 1 -- 172.21.15.7:0/3207278683 learned_addr learned my addr 172.21.15.7:0/3207278683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.176+0000 7fac97400700 1 -- 172.21.15.7:0/3207278683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9810ecf0 con 0x7fac98110040 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.176+0000 7fac97400700 1 --2- 172.21.15.7:0/3207278683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac880098a0 tx=0x7fac88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a628219890e0b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.176+0000 7fac96000700 1 -- 172.21.15.7:0/3207278683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac88004160 con 0x7fac98110040 2024-04-20T21:22:54.407 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.176+0000 7fac96000700 1 -- 172.21.15.7:0/3207278683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac88004300 con 0x7fac98110040 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.176+0000 7fac96000700 1 -- 172.21.15.7:0/3207278683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac88011b20 con 0x7fac98110040 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 -- 172.21.15.7:0/3207278683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 msgr2=0x7fac98110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3207278683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac880098a0 tx=0x7fac88009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 -- 172.21.15.7:0/3207278683 shutdown_connections 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3207278683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac98110460 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 -- 172.21.15.7:0/3207278683 >> 172.21.15.7:0/3207278683 conn(0x7fac9807dc10 msgr2=0x7fac98080070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.177+0000 7fac9ca00700 1 -- 172.21.15.7:0/3207278683 shutdown_connections 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.178+0000 7fac9ca00700 1 -- 172.21.15.7:0/3207278683 wait complete. 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.179+0000 7fac9ca00700 1 Processor -- start 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.179+0000 7fac9ca00700 1 -- start start 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.179+0000 7fac9ca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.179+0000 7fac9ca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac981130b0 con 0x7fac98110040 2024-04-20T21:22:54.408 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.180+0000 7fac97400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.180+0000 7fac97400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39962/0 (socket says 172.21.15.7:39962) 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.180+0000 7fac97400700 1 -- 172.21.15.7:0/3422023064 learned_addr learned my addr 172.21.15.7:0/3422023064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.180+0000 7fac97400700 1 -- 172.21.15.7:0/3422023064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac981a2e50 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.181+0000 7fac97400700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.181+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac88003b60 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.181+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac981a30e0 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.181+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac8801a910 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.181+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac981a3540 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.182+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac88024930 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.182+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fac88022020 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.182+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac980640b0 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.183+0000 7fac8fe00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 0x7fac7c03bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.183+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac8804dae0 con 0x7fac98110040 2024-04-20T21:22:54.409 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.183+0000 7fac96a00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 0x7fac7c03bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.184+0000 7fac96a00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 0x7fac7c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac84006fd0 tx=0x7fac84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.189+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fac8801ab70 con 0x7fac98110040 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.450+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fac981a5e90 con 0x7fac7c0396c0 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.712+0000 7fac8fe00700 1 -- 172.21.15.7:0/3422023064 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fac981a5e90 con 0x7fac7c0396c0 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 msgr2=0x7fac7c03bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 0x7fac7c03bb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac84006fd0 tx=0x7fac84006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 msgr2=0x7fac981a2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac88000c00 tx=0x7fac88011c10 comp rx=0 tx=0).stop 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 shutdown_connections 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac7c0396c0 0x7fac7c03bb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 --2- 172.21.15.7:0/3422023064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac98110040 0x7fac981a2910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.715+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 >> 172.21.15.7:0/3422023064 conn(0x7fac9807dc10 msgr2=0x7fac9807f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:54.411 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.716+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 shutdown_connections 2024-04-20T21:22:54.411 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:53.716+0000 7fac9ca00700 1 -- 172.21.15.7:0/3422023064 wait complete. 2024-04-20T21:22:54.411 INFO:teuthology.orchestra.run.smithi007.stdout:Fetching dashboard port number... 2024-04-20T21:22:54.992 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:54 smithi007 ceph-mon[16867]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:22:54.992 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:22:54.992 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:22:56.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:56 smithi007 ceph-mon[16867]: mgrmap e13: x(active, since 6s) 2024-04-20T21:22:56.133 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:22:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1472747784' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-20T21:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stdout 8443 2024-04-20T21:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.992+0000 7f3d37600700 1 Processor -- start 2024-04-20T21:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.993+0000 7f3d37600700 1 -- start start 2024-04-20T21:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.993+0000 7f3d37600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.895 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.993+0000 7f3d37600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3007c820 con 0x7f3d3007de30 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.994+0000 7f3d34c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.994+0000 7f3d34c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39974/0 (socket says 172.21.15.7:39974) 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.994+0000 7f3d34c00700 1 -- 172.21.15.7:0/2377669927 learned_addr learned my addr 172.21.15.7:0/2377669927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.995+0000 7f3d34c00700 1 -- 172.21.15.7:0/2377669927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3007c9a0 con 0x7f3d3007de30 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.995+0000 7f3d34c00700 1 --2- 172.21.15.7:0/2377669927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d240098a0 tx=0x7f3d24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479d3f4eaadac83f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.995+0000 7f3d2f400700 1 -- 172.21.15.7:0/2377669927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d24004160 con 0x7f3d3007de30 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.995+0000 7f3d2f400700 1 -- 172.21.15.7:0/2377669927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d24004300 con 0x7f3d3007de30 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.995+0000 7f3d2f400700 1 -- 172.21.15.7:0/2377669927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d24011b20 con 0x7f3d3007de30 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 -- 172.21.15.7:0/2377669927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 msgr2=0x7f3d3007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 --2- 172.21.15.7:0/2377669927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d240098a0 tx=0x7f3d24009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 -- 172.21.15.7:0/2377669927 shutdown_connections 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 --2- 172.21.15.7:0/2377669927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3007c2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 -- 172.21.15.7:0/2377669927 >> 172.21.15.7:0/2377669927 conn(0x7f3d30103c10 msgr2=0x7f3d30106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.896 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.996+0000 7f3d37600700 1 -- 172.21.15.7:0/2377669927 shutdown_connections 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.997+0000 7f3d37600700 1 -- 172.21.15.7:0/2377669927 wait complete. 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d37600700 1 Processor -- start 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d37600700 1 -- start start 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d37600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d37600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3010ec30 con 0x7f3d3007de30 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d34c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d34c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39980/0 (socket says 172.21.15.7:39980) 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.998+0000 7f3d34c00700 1 -- 172.21.15.7:0/1472747784 learned_addr learned my addr 172.21.15.7:0/1472747784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.999+0000 7f3d34c00700 1 -- 172.21.15.7:0/1472747784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3019ec70 con 0x7f3d3007de30 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.999+0000 7f3d34c00700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d240117e0 tx=0x7f3d24011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:54.999+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d24003ba0 con 0x7f3d3007de30 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.000+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2401a910 con 0x7f3d3007de30 2024-04-20T21:22:56.897 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.000+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3019ef00 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.000+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d240248f0 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.000+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3019f360 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.001+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f3d24022020 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.001+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d300640b0 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.002+0000 7f3d2d600700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 0x7f3d1803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.002+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d2404e150 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.003+0000 7f3d2fe00700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 0x7f3d1803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.004+0000 7f3d2fe00700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 0x7f3d1803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.008+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3d24020c10 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.019+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f3d2402dc40 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.264+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f3d301a1d30 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.266+0000 7f3d2d600700 1 -- 172.21.15.7:0/1472747784 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f3d2400be60 con 0x7f3d3007de30 2024-04-20T21:22:56.898 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 msgr2=0x7f3d1803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 0x7f3d1803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 msgr2=0x7f3d3019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3d240117e0 tx=0x7f3d24011ba0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 shutdown_connections 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d18039600 0x7f3d1803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 --2- 172.21.15.7:0/1472747784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d3007de30 0x7f3d3019e730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.269+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 >> 172.21.15.7:0/1472747784 conn(0x7f3d30103c10 msgr2=0x7f3d30104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.270+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 shutdown_connections 2024-04-20T21:22:56.899 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:55.270+0000 7f3d37600700 1 -- 172.21.15.7:0/1472747784 wait complete. 2024-04-20T21:22:56.929 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-20T21:22:56.929 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout disabled 2024-04-20T21:22:56.957 INFO:teuthology.orchestra.run.smithi007.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-20T21:22:56.957 INFO:teuthology.orchestra.run.smithi007.stdout:systemctl: stdout inactive 2024-04-20T21:22:56.957 INFO:teuthology.orchestra.run.smithi007.stdout:firewalld.service is not enabled 2024-04-20T21:22:56.957 INFO:teuthology.orchestra.run.smithi007.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-20T21:22:56.964 INFO:teuthology.orchestra.run.smithi007.stdout:Ceph Dashboard is now available at: 2024-04-20T21:22:56.965 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:22:56.965 INFO:teuthology.orchestra.run.smithi007.stdout: URL: https://smithi007.front.sepia.ceph.com:8443/ 2024-04-20T21:22:56.965 INFO:teuthology.orchestra.run.smithi007.stdout: User: admin 2024-04-20T21:22:56.965 INFO:teuthology.orchestra.run.smithi007.stdout: Password: npyuf5qrya 2024-04-20T21:22:56.965 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:22:56.966 INFO:teuthology.orchestra.run.smithi007.stdout:Saving cluster configuration to /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config directory 2024-04-20T21:22:56.968 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling autotune for osd_memory_target 2024-04-20T21:22:58.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.667+0000 7febc0600700 1 Processor -- start 2024-04-20T21:22:58.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.667+0000 7febc0600700 1 -- start start 2024-04-20T21:22:58.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.668+0000 7febc0600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.558 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.668+0000 7febc0600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febb810fa70 con 0x7febb810f080 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.668+0000 7febbdc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.668+0000 7febbdc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39984/0 (socket says 172.21.15.7:39984) 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.668+0000 7febbdc00700 1 -- 172.21.15.7:0/1064977003 learned_addr learned my addr 172.21.15.7:0/1064977003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.669+0000 7febbdc00700 1 -- 172.21.15.7:0/1064977003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb810fe40 con 0x7febb810f080 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.669+0000 7febbdc00700 1 --2- 172.21.15.7:0/1064977003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febac0098a0 tx=0x7febac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45951e9b6bc5b96c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.670+0000 7febb7e00700 1 -- 172.21.15.7:0/1064977003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febac004160 con 0x7febb810f080 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.670+0000 7febb7e00700 1 -- 172.21.15.7:0/1064977003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febac004300 con 0x7febb810f080 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.670+0000 7febb7e00700 1 -- 172.21.15.7:0/1064977003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febac011b20 con 0x7febb810f080 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.670+0000 7febc0600700 1 -- 172.21.15.7:0/1064977003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 msgr2=0x7febb810f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.559 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.670+0000 7febc0600700 1 --2- 172.21.15.7:0/1064977003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febac0098a0 tx=0x7febac009bb0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.671+0000 7febc0600700 1 -- 172.21.15.7:0/1064977003 shutdown_connections 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.671+0000 7febc0600700 1 --2- 172.21.15.7:0/1064977003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb810f4a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.671+0000 7febc0600700 1 -- 172.21.15.7:0/1064977003 >> 172.21.15.7:0/1064977003 conn(0x7febb807de30 msgr2=0x7febb8080270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.671+0000 7febc0600700 1 -- 172.21.15.7:0/1064977003 shutdown_connections 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.671+0000 7febc0600700 1 -- 172.21.15.7:0/1064977003 wait complete. 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.672+0000 7febc0600700 1 Processor -- start 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.672+0000 7febc0600700 1 -- start start 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.672+0000 7febc0600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.672+0000 7febc0600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febb81131f0 con 0x7febb810f080 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.673+0000 7febbdc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.673+0000 7febbdc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39994/0 (socket says 172.21.15.7:39994) 2024-04-20T21:22:58.560 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.673+0000 7febbdc00700 1 -- 172.21.15.7:0/4088038695 learned_addr learned my addr 172.21.15.7:0/4088038695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.673+0000 7febbdc00700 1 -- 172.21.15.7:0/4088038695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb81a3000 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.673+0000 7febbdc00700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febac0116a0 tx=0x7febac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.674+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febac003a40 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.674+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febac003be0 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.674+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febac01a750 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.674+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb81a3290 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.674+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb81a36f0 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.675+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7febac024aa0 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.675+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb80640b0 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.676+0000 7febb6000700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 0x7feba003bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.676+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7febac04de80 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.676+0000 7febbd200700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 0x7feba003bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.677+0000 7febbd200700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 0x7feba003bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feba4006fd0 tx=0x7feba4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.681+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7febac011e20 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.917+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7febb81a6120 con 0x7febb810f080 2024-04-20T21:22:58.561 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.918+0000 7febb6000700 1 -- 172.21.15.7:0/4088038695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7febac023390 con 0x7febb810f080 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 msgr2=0x7feba003bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 0x7feba003bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feba4006fd0 tx=0x7feba4006e40 comp rx=0 tx=0).stop 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 msgr2=0x7febb81a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febac0116a0 tx=0x7febac011ba0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 shutdown_connections 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7feba0039770 0x7feba003bc30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 --2- 172.21.15.7:0/4088038695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb810f080 0x7febb81a2ac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 >> 172.21.15.7:0/4088038695 conn(0x7febb807de30 msgr2=0x7febb807f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 shutdown_connections 2024-04-20T21:22:58.562 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:57.922+0000 7febc0600700 1 -- 172.21.15.7:0/4088038695 wait complete. 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.144+0000 7fbf73000700 1 Processor -- start 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.144+0000 7fbf73000700 1 -- start start 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.145+0000 7fbf73000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.145+0000 7fbf73000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c07c930 con 0x7fbf6c07df00 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.145+0000 7fbf6be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.208 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.145+0000 7fbf6be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40000/0 (socket says 172.21.15.7:40000) 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.145+0000 7fbf6be00700 1 -- 172.21.15.7:0/1864933799 learned_addr learned my addr 172.21.15.7:0/1864933799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.146+0000 7fbf6be00700 1 -- 172.21.15.7:0/1864933799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c07cab0 con 0x7fbf6c07df00 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.146+0000 7fbf6be00700 1 --2- 172.21.15.7:0/1864933799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098a0 tx=0x7fbf5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f61a529c9e9d3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf6b400700 1 -- 172.21.15.7:0/1864933799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c004160 con 0x7fbf6c07df00 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf6b400700 1 -- 172.21.15.7:0/1864933799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5c004300 con 0x7fbf6c07df00 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf6b400700 1 -- 172.21.15.7:0/1864933799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c011b20 con 0x7fbf6c07df00 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf73000700 1 -- 172.21.15.7:0/1864933799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 msgr2=0x7fbf6c07c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf73000700 1 --2- 172.21.15.7:0/1864933799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098a0 tx=0x7fbf5c009bb0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.209 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf73000700 1 -- 172.21.15.7:0/1864933799 shutdown_connections 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf73000700 1 --2- 172.21.15.7:0/1864933799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c07c360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.147+0000 7fbf73000700 1 -- 172.21.15.7:0/1864933799 >> 172.21.15.7:0/1864933799 conn(0x7fbf6c103be0 msgr2=0x7fbf6c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.148+0000 7fbf73000700 1 -- 172.21.15.7:0/1864933799 shutdown_connections 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.148+0000 7fbf73000700 1 -- 172.21.15.7:0/1864933799 wait complete. 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.149+0000 7fbf73000700 1 Processor -- start 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.149+0000 7fbf73000700 1 -- start start 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.149+0000 7fbf73000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.149+0000 7fbf73000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c10a860 con 0x7fbf6c07df00 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf6be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf6be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40004/0 (socket says 172.21.15.7:40004) 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf6be00700 1 -- 172.21.15.7:0/3338460246 learned_addr learned my addr 172.21.15.7:0/3338460246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf6be00700 1 -- 172.21.15.7:0/3338460246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c1a3060 con 0x7fbf6c07df00 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf6be00700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0116a0 tx=0x7fbf5c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.210 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.150+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c003a40 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.151+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf5c003be0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.151+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c1a32f0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.151+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c01a5e0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.151+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1a3750 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.152+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fbf5c0249a0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.152+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0640b0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.152+0000 7fbf69600700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 0x7fbf5003bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.152+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbf5c04de40 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.153+0000 7fbf63e00700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 0x7fbf5003bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.154+0000 7fbf63e00700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 0x7fbf5003bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf54006fd0 tx=0x7fbf54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.159+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf5c011e20 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.509+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fbf6c19c0a0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.514+0000 7fbf69600700 1 -- 172.21.15.7:0/3338460246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fbf6c19c0a0 con 0x7fbf6c07df00 2024-04-20T21:23:00.211 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 msgr2=0x7fbf5003bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 0x7fbf5003bbd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbf54006fd0 tx=0x7fbf54006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 msgr2=0x7fbf6c1a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0116a0 tx=0x7fbf5c011ba0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 shutdown_connections 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf50039710 0x7fbf5003bbd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 --2- 172.21.15.7:0/3338460246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6c07df00 0x7fbf6c1a2b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 >> 172.21.15.7:0/3338460246 conn(0x7fbf6c103be0 msgr2=0x7fbf6c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.517+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 shutdown_connections 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr 2024-04-20T21:22:59.518+0000 7fbf73000700 1 -- 172.21.15.7:0/3338460246 wait complete. 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.212 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout:Or, if you are only running a single cluster on this host: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: ceph telemetry on 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout:For more information see: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:00.213 INFO:teuthology.orchestra.run.smithi007.stdout:Bootstrap complete. 2024-04-20T21:23:00.284 INFO:tasks.cephadm:Fetching config... 2024-04-20T21:23:00.285 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:23:00.285 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-20T21:23:00.331 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-20T21:23:00.331 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:23:00.331 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-20T21:23:00.423 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-20T21:23:00.423 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:23:00.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-20T21:23:00.549 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-20T21:23:00.549 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:23:00.549 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-20T21:23:00.599 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-20T21:23:00.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:00.763 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:00 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3338460246' entity='client.admin' 2024-04-20T21:23:00.771 INFO:teuthology.orchestra.run.smithi007.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:00.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:00.878 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-20T21:23:00.917 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSs5AntvOueIwyM45gT7ty0Xu8QvF1MfbTLGbfE9fsV28/q7eKBRrdPXFk1qwZqVzLyKP0dyRW6QKaqoq1FB7OFH+afriXBpE2T4Q8FHdNMjKcveCMY1DHPIhBbh9Ny+DT2ybe9NohdRvA5T3MuvDKEhXzCFdCIL8M+Cl72bT4lpgsi9NNfjk2HCJTKPg7K40UUdqxnQKQSsXNBxz0xGvi2AhRcnrvGITXfsJB6LCLPgavibvRngZaJ891mxhksqn5U3cfNqv3N8I73bEWZZdVhIhrxEH254naNN1lOumdOWXM8yf6MSBNSEauGscNnZ51zA9TR4hkoVirnF1UT2eSsfcgJGSr6ybSs8muz5oiD925egVGjItoJEt6PPAEJ7WnSpGJdoICT6Ad9s+wjdgMQraUiYt5oqG0xi6sez0RDBbgRO4qDd9rRODPoR04Q5rjHxeh9fOV1rxMOMNsqJpAoOtQIAWxeJVOuE4r9q1WQKrIiqcoupBoylMelUYqLus= ceph-8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:00.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-20T21:23:01.366 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 -- 172.21.15.7:0/2864736021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 msgr2=0x7f49d0109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 --2- 172.21.15.7:0/2864736021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d0109080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009910 tx=0x7f49c4009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 -- 172.21.15.7:0/2864736021 shutdown_connections 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 --2- 172.21.15.7:0/2864736021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d0109080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 -- 172.21.15.7:0/2864736021 >> 172.21.15.7:0/2864736021 conn(0x7f49d0077b40 msgr2=0x7f49d0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 -- 172.21.15.7:0/2864736021 shutdown_connections 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.684+0000 7f49d8800700 1 -- 172.21.15.7:0/2864736021 wait complete. 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.685+0000 7f49d8800700 1 Processor -- start 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.685+0000 7f49d8800700 1 -- start start 2024-04-20T21:23:03.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d8800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d8800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49d010eb80 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43310/0 (socket says 172.21.15.7:43310) 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d5e00700 1 -- 172.21.15.7:0/2067726928 learned_addr learned my addr 172.21.15.7:0/2067726928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.686+0000 7f49d5e00700 1 -- 172.21.15.7:0/2067726928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d019e730 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49d5e00700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005850 tx=0x7f49c4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4026070 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c40044a0 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d019e9c0 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49c4003ab0 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.687+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a16b0 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.688+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f49c4029070 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.688+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d010f300 con 0x7f49d0108c60 2024-04-20T21:23:03.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.689+0000 7f49caa00700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 0x7f49b803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.689+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f49c401e070 con 0x7f49d0108c60 2024-04-20T21:23:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.689+0000 7f49d5400700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 0x7f49b803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.690+0000 7f49d5400700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 0x7f49b803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.695+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f49c4013070 con 0x7f49d0108c60 2024-04-20T21:23:03.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.932+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f49d002e0a0 con 0x7f49d0108c60 2024-04-20T21:23:03.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.936+0000 7f49caa00700 1 -- 172.21.15.7:0/2067726928 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f49c40300c0 con 0x7f49d0108c60 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 msgr2=0x7f49b803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 0x7f49b803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 msgr2=0x7f49d019e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005850 tx=0x7f49c4005030 comp rx=0 tx=0).stop 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 shutdown_connections 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f49b80396b0 0x7f49b803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 --2- 172.21.15.7:0/2067726928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d0108c60 0x7f49d019e1f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 >> 172.21.15.7:0/2067726928 conn(0x7f49d0077b40 msgr2=0x7f49d0078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.940+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 shutdown_connections 2024-04-20T21:23:03.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:03.941+0000 7f49d8800700 1 -- 172.21.15.7:0/2067726928 wait complete. 2024-04-20T21:23:04.633 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-20T21:23:04.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-20T21:23:05.082 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:05.235 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2067726928' entity='client.admin' 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi007", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:05.236 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 -- 172.21.15.7:0/646916481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 msgr2=0x7fb360103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 --2- 172.21.15.7:0/646916481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb360103160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb35000b0a0 tx=0x7fb35000b3b0 comp rx=0 tx=0).stop 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 -- 172.21.15.7:0/646916481 shutdown_connections 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 --2- 172.21.15.7:0/646916481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb360103160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 -- 172.21.15.7:0/646916481 >> 172.21.15.7:0/646916481 conn(0x7fb3600fc950 msgr2=0x7fb3600fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 -- 172.21.15.7:0/646916481 shutdown_connections 2024-04-20T21:23:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.079+0000 7fb365000700 1 -- 172.21.15.7:0/646916481 wait complete. 2024-04-20T21:23:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.080+0000 7fb365000700 1 Processor -- start 2024-04-20T21:23:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.080+0000 7fb365000700 1 -- start start 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb365000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb365000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3601063f0 con 0x7fb360100d30 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb35de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb35de00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43320/0 (socket says 172.21.15.7:43320) 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb35de00700 1 -- 172.21.15.7:0/3228159425 learned_addr learned my addr 172.21.15.7:0/3228159425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.081+0000 7fb35de00700 1 -- 172.21.15.7:0/3228159425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36019e620 con 0x7fb360100d30 2024-04-20T21:23:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb35de00700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb350015040 tx=0x7fb350009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb350017040 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb350022070 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36019e8b0 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb35001d5d0 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.082+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3601a15a0 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.083+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb360106b70 con 0x7fb360100d30 2024-04-20T21:23:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.084+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fb350004000 con 0x7fb360100d30 2024-04-20T21:23:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.084+0000 7fb357400700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 0x7fb34403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.084+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb35000f050 con 0x7fb360100d30 2024-04-20T21:23:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.085+0000 7fb355400700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 0x7fb34403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.085+0000 7fb355400700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 0x7fb34403bb60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb348006fd0 tx=0x7fb348006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.090+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb350054080 con 0x7fb360100d30 2024-04-20T21:23:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.326+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb36002e0a0 con 0x7fb3440396a0 2024-04-20T21:23:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.333+0000 7fb357400700 1 -- 172.21.15.7:0/3228159425 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb36002e0a0 con 0x7fb3440396a0 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 msgr2=0x7fb34403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 0x7fb34403bb60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb348006fd0 tx=0x7fb348006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 msgr2=0x7fb36019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb350015040 tx=0x7fb350009790 comp rx=0 tx=0).stop 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 shutdown_connections 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb3440396a0 0x7fb34403bb60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:07.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 --2- 172.21.15.7:0/3228159425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb360100d30 0x7fb36019e0e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:07.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 >> 172.21.15.7:0/3228159425 conn(0x7fb3600fc950 msgr2=0x7fb3600fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:07.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 shutdown_connections 2024-04-20T21:23:07.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:07.336+0000 7fb365000700 1 -- 172.21.15.7:0/3228159425 wait complete. 2024-04-20T21:23:08.059 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi016 2024-04-20T21:23:08.060 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:23:08.060 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.conf 2024-04-20T21:23:08.078 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:23:08.078 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:08.136 INFO:tasks.cephadm:Adding host smithi016 to orchestrator... 2024-04-20T21:23:08.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch host add smithi016 2024-04-20T21:23:08.556 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:08.702 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:08.702 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:08.702 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:08.702 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:08.702 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:08.703 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:23:08.703 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:08 smithi007 ceph-mon[16867]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:09.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:09 smithi007 ceph-mon[16867]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:09.738 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:09 smithi007 ceph-mon[16867]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:10.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 -- 172.21.15.7:0/1265784107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751010a660 msgr2=0x7f751010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 --2- 172.21.15.7:0/1265784107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751010a660 0x7f751010aa40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7500009910 tx=0x7f7500009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 -- 172.21.15.7:0/1265784107 shutdown_connections 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 --2- 172.21.15.7:0/1265784107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751010a660 0x7f751010aa40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 -- 172.21.15.7:0/1265784107 >> 172.21.15.7:0/1265784107 conn(0x7f7510100170 msgr2=0x7f7510102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 -- 172.21.15.7:0/1265784107 shutdown_connections 2024-04-20T21:23:10.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.594+0000 7f7515800700 1 -- 172.21.15.7:0/1265784107 wait complete. 2024-04-20T21:23:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.595+0000 7f7515800700 1 Processor -- start 2024-04-20T21:23:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.595+0000 7f7515800700 1 -- start start 2024-04-20T21:23:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.596+0000 7f7515800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.596+0000 7f7515800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f751010eca0 con 0x7f751019a3d0 2024-04-20T21:23:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.596+0000 7f750ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.596+0000 7f750ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43334/0 (socket says 172.21.15.7:43334) 2024-04-20T21:23:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.596+0000 7f750ea00700 1 -- 172.21.15.7:0/3575576553 learned_addr learned my addr 172.21.15.7:0/3575576553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f750ea00700 1 -- 172.21.15.7:0/3575576553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751019acf0 con 0x7f751019a3d0 2024-04-20T21:23:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f750ea00700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f750000b5c0 tx=0x7f7500005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7500026070 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f750000b6f0 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75000037f0 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751019afe0 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.597+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751019ee20 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.598+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7510105da0 con 0x7f751019a3d0 2024-04-20T21:23:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.599+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f7500029070 con 0x7f751019a3d0 2024-04-20T21:23:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.599+0000 7f7507400700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 0x7f74f403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.599+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f750001e070 con 0x7f751019a3d0 2024-04-20T21:23:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.599+0000 7f7507e00700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 0x7f74f403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.600+0000 7f7507e00700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 0x7f74f403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006fd0 tx=0x7f74f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.605+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7500013070 con 0x7f751019a3d0 2024-04-20T21:23:10.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:10.842+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}) -- 0x7f751019b6e0 con 0x7f74f40396b0 2024-04-20T21:23:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:11 smithi007 ceph-mon[16867]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:12.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:12 smithi007 ceph-mon[16867]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:12.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:12 smithi007 ceph-mon[16867]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:13 smithi007 ceph-mon[16867]: Deploying cephadm binary to smithi016 2024-04-20T21:23:14.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:14 smithi007 ceph-mon[16867]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:15.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.795+0000 7f7507400700 1 -- 172.21.15.7:0/3575576553 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f751019b6e0 con 0x7f74f40396b0 2024-04-20T21:23:15.796 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi016' with addr '172.21.15.16' 2024-04-20T21:23:15.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 msgr2=0x7f74f403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 0x7f74f403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006fd0 tx=0x7f74f8006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:15.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 msgr2=0x7f751019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f750000b5c0 tx=0x7f7500005030 comp rx=0 tx=0).stop 2024-04-20T21:23:15.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 shutdown_connections 2024-04-20T21:23:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74f40396b0 0x7f74f403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 --2- 172.21.15.7:0/3575576553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f751019a3d0 0x7f751019a7b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 >> 172.21.15.7:0/3575576553 conn(0x7f7510100170 msgr2=0x7f751010b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 shutdown_connections 2024-04-20T21:23:15.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:15.799+0000 7f7515800700 1 -- 172.21.15.7:0/3575576553 wait complete. 2024-04-20T21:23:16.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:16 smithi007 ceph-mon[16867]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:16.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:16.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:16.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T21:23:16.973 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:17 smithi007 ceph-mon[16867]: Added host smithi016 2024-04-20T21:23:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:17 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:17 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:18.181 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:18 smithi007 ceph-mon[16867]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 -- 172.21.15.7:0/896753864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 msgr2=0x7f705c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 --2- 172.21.15.7:0/896753864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c10dd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f704c009910 tx=0x7f704c009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 -- 172.21.15.7:0/896753864 shutdown_connections 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 --2- 172.21.15.7:0/896753864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c10dd50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 -- 172.21.15.7:0/896753864 >> 172.21.15.7:0/896753864 conn(0x7f705c0781f0 msgr2=0x7f705c07a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 -- 172.21.15.7:0/896753864 shutdown_connections 2024-04-20T21:23:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.958+0000 7f7062e00700 1 -- 172.21.15.7:0/896753864 wait complete. 2024-04-20T21:23:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.959+0000 7f7062e00700 1 Processor -- start 2024-04-20T21:23:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f7062e00700 1 -- start start 2024-04-20T21:23:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f7062e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f7062e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1130c0 con 0x7f705c10d970 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f705be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f705be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41378/0 (socket says 172.21.15.7:41378) 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.960+0000 7f705be00700 1 -- 172.21.15.7:0/3921628059 learned_addr learned my addr 172.21.15.7:0/3921628059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f705be00700 1 -- 172.21.15.7:0/3921628059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c19ea20 con 0x7f705c10d970 2024-04-20T21:23:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f705be00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f704c006010 tx=0x7f704c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f704c026070 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c00bbd0 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c19ed10 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f704c0039d0 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.961+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a2db0 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.963+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f704c029070 con 0x7f705c10d970 2024-04-20T21:23:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.963+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c10a190 con 0x7f705c10d970 2024-04-20T21:23:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.963+0000 7f7058c00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 0x7f704003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.963+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f704c01e070 con 0x7f705c10d970 2024-04-20T21:23:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.964+0000 7f705b400700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 0x7f704003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.964+0000 7f705b400700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 0x7f704003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7048006fd0 tx=0x7f7048006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:18.969+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f704c013070 con 0x7f705c10d970 2024-04-20T21:23:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.207+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f705c19f840 con 0x7f7040039650 2024-04-20T21:23:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.210+0000 7f7058c00700 1 -- 172.21.15.7:0/3921628059 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f705c19f840 con 0x7f7040039650 2024-04-20T21:23:19.210 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:19.210 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}] 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 msgr2=0x7f704003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 0x7f704003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7048006fd0 tx=0x7f7048006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 msgr2=0x7f705c19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f704c006010 tx=0x7f704c005030 comp rx=0 tx=0).stop 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 shutdown_connections 2024-04-20T21:23:19.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7040039650 0x7f704003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 --2- 172.21.15.7:0/3921628059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f705c10d970 0x7f705c19e4e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 >> 172.21.15.7:0/3921628059 conn(0x7f705c0781f0 msgr2=0x7f705c1104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 shutdown_connections 2024-04-20T21:23:19.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:19.213+0000 7f7062e00700 1 -- 172.21.15.7:0/3921628059 wait complete. 2024-04-20T21:23:19.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:19 smithi007 ceph-mon[16867]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:19.878 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2024-04-20T21:23:19.878 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:23:19.878 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2024-04-20T21:23:19.897 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:23:19.897 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:19.956 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2024-04-20T21:23:19.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch host add smithi186 2024-04-20T21:23:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:20.816 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:20 smithi007 ceph-mon[16867]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:21.925 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:21 smithi007 ceph-mon[16867]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 -- 172.21.15.7:0/2497462049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 msgr2=0x7f52301088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 --2- 172.21.15.7:0/2497462049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f52301088d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5220009910 tx=0x7f5220009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 -- 172.21.15.7:0/2497462049 shutdown_connections 2024-04-20T21:23:22.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 --2- 172.21.15.7:0/2497462049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f52301088d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 -- 172.21.15.7:0/2497462049 >> 172.21.15.7:0/2497462049 conn(0x7f52300fdf90 msgr2=0x7f52301003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 -- 172.21.15.7:0/2497462049 shutdown_connections 2024-04-20T21:23:22.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.426+0000 7f5235800700 1 -- 172.21.15.7:0/2497462049 wait complete. 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.427+0000 7f5235800700 1 Processor -- start 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.427+0000 7f5235800700 1 -- start start 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f5235800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f5235800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f523010caa0 con 0x7f52301084f0 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f522ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f522ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41400/0 (socket says 172.21.15.7:41400) 2024-04-20T21:23:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f522ea00700 1 -- 172.21.15.7:0/3491890777 learned_addr learned my addr 172.21.15.7:0/3491890777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.428+0000 7f522ea00700 1 -- 172.21.15.7:0/3491890777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52301984f0 con 0x7f52301084f0 2024-04-20T21:23:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.429+0000 7f522ea00700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5220006010 tx=0x7f5220005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.429+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5220026070 con 0x7f52301084f0 2024-04-20T21:23:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.429+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52301987e0 con 0x7f52301084f0 2024-04-20T21:23:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.429+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522000bbd0 con 0x7f52301084f0 2024-04-20T21:23:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.429+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523019c880 con 0x7f52301084f0 2024-04-20T21:23:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.430+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5220030060 con 0x7f52301084f0 2024-04-20T21:23:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.430+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f5220029070 con 0x7f52301084f0 2024-04-20T21:23:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.430+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5230103ba0 con 0x7f52301084f0 2024-04-20T21:23:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.431+0000 7f5227e00700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 0x7f521403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.431+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f522001e070 con 0x7f52301084f0 2024-04-20T21:23:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.432+0000 7f5226000700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 0x7f521403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.432+0000 7f5226000700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 0x7f521403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5218006fd0 tx=0x7f5218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.437+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5220013070 con 0x7f52301084f0 2024-04-20T21:23:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:22.672+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) -- 0x7f5230199310 con 0x7f52140396b0 2024-04-20T21:23:23.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:23.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:23.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:23.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:23.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:23.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:23.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:23.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:22 smithi007 ceph-mon[16867]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:24.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:24.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:25.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:24 smithi007 ceph-mon[16867]: Deploying cephadm binary to smithi186 2024-04-20T21:23:26.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:25 smithi007 ceph-mon[16867]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.593+0000 7f5227e00700 1 -- 172.21.15.7:0/3491890777 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5230199310 con 0x7f52140396b0 2024-04-20T21:23:27.592 INFO:teuthology.orchestra.run.smithi007.stdout:Added host 'smithi186' with addr '172.21.15.186' 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 msgr2=0x7f521403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 0x7f521403bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5218006fd0 tx=0x7f5218006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 msgr2=0x7f5230197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5220006010 tx=0x7f5220005030 comp rx=0 tx=0).stop 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 shutdown_connections 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52140396b0 0x7f521403bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 --2- 172.21.15.7:0/3491890777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52301084f0 0x7f5230197fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.596+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 >> 172.21.15.7:0/3491890777 conn(0x7f52300fdf90 msgr2=0x7f5230109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.597+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 shutdown_connections 2024-04-20T21:23:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:27.597+0000 7f5235800700 1 -- 172.21.15.7:0/3491890777 wait complete. 2024-04-20T21:23:27.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:27 smithi007 ceph-mon[16867]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:27.972 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:27.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:28.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-20T21:23:28.762 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:29.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:28 smithi007 ceph-mon[16867]: Added host smithi186 2024-04-20T21:23:29.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:28 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:29.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:28 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:29.922 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:29 smithi007 ceph-mon[16867]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:30.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 -- 172.21.15.7:0/2895749019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 msgr2=0x7ff9b810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 --2- 172.21.15.7:0/2895749019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b810dd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009910 tx=0x7ff9a8009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 -- 172.21.15.7:0/2895749019 shutdown_connections 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 --2- 172.21.15.7:0/2895749019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b810dd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 -- 172.21.15.7:0/2895749019 >> 172.21.15.7:0/2895749019 conn(0x7ff9b80781f0 msgr2=0x7ff9b807a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 -- 172.21.15.7:0/2895749019 shutdown_connections 2024-04-20T21:23:30.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.693+0000 7ff9be200700 1 -- 172.21.15.7:0/2895749019 wait complete. 2024-04-20T21:23:30.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.694+0000 7ff9be200700 1 Processor -- start 2024-04-20T21:23:30.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.694+0000 7ff9be200700 1 -- start start 2024-04-20T21:23:30.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.695+0000 7ff9be200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.695+0000 7ff9be200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b81130c0 con 0x7ff9b810d970 2024-04-20T21:23:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.695+0000 7ff9b7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.695+0000 7ff9b7400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60420/0 (socket says 172.21.15.7:60420) 2024-04-20T21:23:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.695+0000 7ff9b7400700 1 -- 172.21.15.7:0/2642758723 learned_addr learned my addr 172.21.15.7:0/2642758723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9b7400700 1 -- 172.21.15.7:0/2642758723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b819ea20 con 0x7ff9b810d970 2024-04-20T21:23:30.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9b7400700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8006010 tx=0x7ff9a8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a8026070 con 0x7ff9b810d970 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a800bbd0 con 0x7ff9b810d970 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9a8003a30 con 0x7ff9b810d970 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.696+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b819ed10 con 0x7ff9b810d970 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.697+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b81a2db0 con 0x7ff9b810d970 2024-04-20T21:23:30.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.698+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b810a190 con 0x7ff9b810d970 2024-04-20T21:23:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.698+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7ff9a8029070 con 0x7ff9b810d970 2024-04-20T21:23:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.698+0000 7ff9afe00700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 0x7ff99c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:30.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.699+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff9a801e070 con 0x7ff9b810d970 2024-04-20T21:23:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.699+0000 7ff9b6a00700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 0x7ff99c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:30.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.699+0000 7ff9b6a00700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 0x7ff99c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4006fd0 tx=0x7ff9a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:30.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.704+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff9a8013070 con 0x7ff9b810d970 2024-04-20T21:23:30.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.943+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9b819f840 con 0x7ff99c039650 2024-04-20T21:23:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.946+0000 7ff9afe00700 1 -- 172.21.15.7:0/2642758723 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7ff9b819f840 con 0x7ff99c039650 2024-04-20T21:23:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:23:30.945 INFO:teuthology.orchestra.run.smithi007.stdout:[{"addr": "172.21.15.7", "hostname": "smithi007", "labels": [], "status": ""}, {"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2024-04-20T21:23:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 msgr2=0x7ff99c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 0x7ff99c03bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4006fd0 tx=0x7ff9a4006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 msgr2=0x7ff9b819e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8006010 tx=0x7ff9a8005050 comp rx=0 tx=0).stop 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 shutdown_connections 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff99c039650 0x7ff99c03bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 --2- 172.21.15.7:0/2642758723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b810d970 0x7ff9b819e4e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 >> 172.21.15.7:0/2642758723 conn(0x7ff9b80781f0 msgr2=0x7ff9b81104d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.949+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 shutdown_connections 2024-04-20T21:23:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:30.950+0000 7ff9be200700 1 -- 172.21.15.7:0/2642758723 wait complete. 2024-04-20T21:23:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:31 smithi007 ceph-mon[16867]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:31 smithi007 ceph-mon[16867]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:31.650 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-20T21:23:31.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-20T21:23:32.103 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:33.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:33 smithi007 ceph-mon[16867]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.110+0000 7f1cca400700 1 -- 172.21.15.7:0/3621039183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 msgr2=0x7f1cc41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.110+0000 7f1cca400700 1 --2- 172.21.15.7:0/3621039183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc41028e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009910 tx=0x7f1cb8009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.111+0000 7f1cca400700 1 -- 172.21.15.7:0/3621039183 shutdown_connections 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.111+0000 7f1cca400700 1 --2- 172.21.15.7:0/3621039183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc41028e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.111+0000 7f1cca400700 1 -- 172.21.15.7:0/3621039183 >> 172.21.15.7:0/3621039183 conn(0x7f1cc40fddb0 msgr2=0x7f1cc41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.111+0000 7f1cca400700 1 -- 172.21.15.7:0/3621039183 shutdown_connections 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.111+0000 7f1cca400700 1 -- 172.21.15.7:0/3621039183 wait complete. 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.112+0000 7f1cca400700 1 Processor -- start 2024-04-20T21:23:34.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.112+0000 7f1cca400700 1 -- start start 2024-04-20T21:23:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.113+0000 7f1cca400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.113+0000 7f1cca400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410c8d0 con 0x7f1cc4102500 2024-04-20T21:23:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.113+0000 7f1cc9000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.113+0000 7f1cc9000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47774/0 (socket says 172.21.15.7:47774) 2024-04-20T21:23:34.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.113+0000 7f1cc9000700 1 -- 172.21.15.7:0/467024116 learned_addr learned my addr 172.21.15.7:0/467024116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cc9000700 1 -- 172.21.15.7:0/467024116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc4198310 con 0x7f1cc4102500 2024-04-20T21:23:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cc9000700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8026070 con 0x7f1cc4102500 2024-04-20T21:23:34.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb800bbd0 con 0x7f1cc4102500 2024-04-20T21:23:34.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc4198600 con 0x7f1cc4102500 2024-04-20T21:23:34.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.114+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cb8003910 con 0x7f1cc4102500 2024-04-20T21:23:34.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.115+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc419c6a0 con 0x7f1cc4102500 2024-04-20T21:23:34.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.116+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc41039e0 con 0x7f1cc4102500 2024-04-20T21:23:34.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.116+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f1cb8029070 con 0x7f1cc4102500 2024-04-20T21:23:34.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.117+0000 7f1cc1600700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 0x7f1cac03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:34.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.117+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1cb801e070 con 0x7f1cc4102500 2024-04-20T21:23:34.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.117+0000 7f1cc3e00700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 0x7f1cac03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:34.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.118+0000 7f1cc3e00700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 0x7f1cac03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:34.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.123+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1cb8013070 con 0x7f1cc4102500 2024-04-20T21:23:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.364+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1cc40640b0 con 0x7f1cc4102500 2024-04-20T21:23:34.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.553+0000 7f1cc1600700 1 -- 172.21.15.7:0/467024116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1cb800fe20 con 0x7f1cc4102500 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.557+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 msgr2=0x7f1cac03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.557+0000 7f1cca400700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 0x7f1cac03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0006fd0 tx=0x7f1cb0006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.557+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 msgr2=0x7f1cc4197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.557+0000 7f1cca400700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8006010 tx=0x7f1cb8005030 comp rx=0 tx=0).stop 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 shutdown_connections 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1cac0396b0 0x7f1cac03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 --2- 172.21.15.7:0/467024116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4102500 0x7f1cc4197dd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 >> 172.21.15.7:0/467024116 conn(0x7f1cc40fddb0 msgr2=0x7f1cc4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 shutdown_connections 2024-04-20T21:23:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:34.558+0000 7f1cca400700 1 -- 172.21.15.7:0/467024116 wait complete. 2024-04-20T21:23:34.558 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T21:23:35.271 INFO:tasks.cephadm:Adding mon.a on smithi007 2024-04-20T21:23:35.272 INFO:tasks.cephadm:Adding mon.b on smithi016 2024-04-20T21:23:35.272 INFO:tasks.cephadm:Adding mon.c on smithi186 2024-04-20T21:23:35.272 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c' 2024-04-20T21:23:35.497 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:36.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:36.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:36.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T21:23:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:36.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:37.017 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.017+0000 7f6334a0f700 1 -- 172.21.15.186:0/169877626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 msgr2=0x7f63301088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.017+0000 7f6334a0f700 1 --2- 172.21.15.186:0/169877626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f63301088d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6324009910 tx=0x7f6324009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.017+0000 7f6334a0f700 1 -- 172.21.15.186:0/169877626 shutdown_connections 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.017+0000 7f6334a0f700 1 --2- 172.21.15.186:0/169877626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f63301088d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.017+0000 7f6334a0f700 1 -- 172.21.15.186:0/169877626 >> 172.21.15.186:0/169877626 conn(0x7f63300fdf90 msgr2=0x7f63301003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 -- 172.21.15.186:0/169877626 shutdown_connections 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 -- 172.21.15.186:0/169877626 wait complete. 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 Processor -- start 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 -- start start 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:37.018 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.018+0000 7f6334a0f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f633010caa0 con 0x7f63301084f0 2024-04-20T21:23:37.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.019+0000 7f632e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:37.019 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.019+0000 7f632e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.186:38918/0 (socket says 172.21.15.186:38918) 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.019+0000 7f632e59c700 1 -- 172.21.15.186:0/1131744185 learned_addr learned my addr 172.21.15.186:0/1131744185 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f632e59c700 1 -- 172.21.15.186:0/1131744185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63301984f0 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f632e59c700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6324005850 tx=0x7f6324005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6324026070 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63301987e0 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633019c880 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632400bbd0 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.020+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63240039b0 con 0x7f63301084f0 2024-04-20T21:23:37.020 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.021+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6330103ba0 con 0x7f63301084f0 2024-04-20T21:23:37.021 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.022+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f6324029070 con 0x7f63301084f0 2024-04-20T21:23:37.021 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.022+0000 7f631f7fe700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 0x7f631803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:37.021 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.022+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f632401e070 con 0x7f63301084f0 2024-04-20T21:23:37.022 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.023+0000 7f632dd9b700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 0x7f631803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:37.023 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.024+0000 7f632dd9b700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 0x7f631803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6320006fd0 tx=0x7f6320006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:37.035 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.036+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6324013070 con 0x7f63301084f0 2024-04-20T21:23:37.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.245+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}) -- 0x7f6330199310 con 0x7f6318039650 2024-04-20T21:23:37.260 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.261+0000 7f631f7fe700 1 -- 172.21.15.186:0/1131744185 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6330199310 con 0x7f6318039650 2024-04-20T21:23:37.260 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mon update... 2024-04-20T21:23:37.262 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 msgr2=0x7f631803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.262 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 0x7f631803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6320006fd0 tx=0x7f6320006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:37.262 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 msgr2=0x7f6330197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6324005850 tx=0x7f6324005030 comp rx=0 tx=0).stop 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 shutdown_connections 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6318039650 0x7f631803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 --2- 172.21.15.186:0/1131744185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63301084f0 0x7f6330197fb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 >> 172.21.15.186:0/1131744185 conn(0x7f63300fdf90 msgr2=0x7f6330109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 shutdown_connections 2024-04-20T21:23:37.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:37.263+0000 7f6334a0f700 1 -- 172.21.15.186:0/1131744185 wait complete. 2024-04-20T21:23:37.877 DEBUG:teuthology.orchestra.run.smithi016:mon.b> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.b.service 2024-04-20T21:23:37.879 DEBUG:teuthology.orchestra.run.smithi186:mon.c> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.c.service 2024-04-20T21:23:37.882 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:23:37.882 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:23:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:38.646 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:38.889 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:38 smithi007 ceph-mon[16867]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:38.889 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:38 smithi007 ceph-mon[16867]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c;count:3 2024-04-20T21:23:38.889 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:38 smithi007 ceph-mon[16867]: Deploying daemon mon.c on smithi186 2024-04-20T21:23:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:39 smithi007 ceph-mon[16867]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 -- 172.21.15.186:0/1017018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 msgr2=0x7f06d8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 --2- 172.21.15.186:0/1017018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8108940 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009910 tx=0x7f06d4009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 -- 172.21.15.186:0/1017018353 shutdown_connections 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 --2- 172.21.15.186:0/1017018353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8108940 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.317 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 -- 172.21.15.186:0/1017018353 >> 172.21.15.186:0/1017018353 conn(0x7f06d80fdfa0 msgr2=0x7f06d81003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 -- 172.21.15.186:0/1017018353 shutdown_connections 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.317+0000 7f06e0369700 1 -- 172.21.15.186:0/1017018353 wait complete. 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.318+0000 7f06e0369700 1 Processor -- start 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.318+0000 7f06e0369700 1 -- start start 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.318+0000 7f06e0369700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.318 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.318+0000 7f06e0369700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810cb20 con 0x7f06d8108560 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.318+0000 7f06de105700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.319+0000 7f06de105700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.186:48762/0 (socket says 172.21.15.186:48762) 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.319+0000 7f06de105700 1 -- 172.21.15.186:0/1590799670 learned_addr learned my addr 172.21.15.186:0/1590799670 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.319+0000 7f06de105700 1 -- 172.21.15.186:0/1590799670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d8198440 con 0x7f06d8108560 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.319+0000 7f06de105700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005850 tx=0x7f06d4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.319 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.320+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d4026070 con 0x7f06d8108560 2024-04-20T21:23:41.320 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.320+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81986d0 con 0x7f06d8108560 2024-04-20T21:23:41.320 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.320+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d819c7d0 con 0x7f06d8108560 2024-04-20T21:23:41.320 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.320+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d400b750 con 0x7f06d8108560 2024-04-20T21:23:41.320 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.320+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06d4030060 con 0x7f06d8108560 2024-04-20T21:23:41.320 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.321+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06bc005360 con 0x7f06d8108560 2024-04-20T21:23:41.321 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.321+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f06d4029070 con 0x7f06d8108560 2024-04-20T21:23:41.321 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.321+0000 7f06cf7fe700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 0x7f06c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:41.321 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.321+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f06d401e070 con 0x7f06d8108560 2024-04-20T21:23:41.321 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.322+0000 7f06dd904700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 0x7f06c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:41.322 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.323+0000 7f06dd904700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 0x7f06c403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:41.336 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.337+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f06d4013070 con 0x7f06d8108560 2024-04-20T21:23:41.633 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.634+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f06bc005650 con 0x7f06d8108560 2024-04-20T21:23:41.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.635+0000 7f06cf7fe700 1 -- 172.21.15.186:0/1590799670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f06d404bf80 con 0x7f06d8108560 2024-04-20T21:23:41.634 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:23:41.634 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":1,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:21:28.997163Z","created":"2024-04-20T21:21:28.997163Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 msgr2=0x7f06c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 0x7f06c403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f06c8006fd0 tx=0x7f06c8006e40 comp rx=0 tx=0).stop 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 msgr2=0x7f06d8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005850 tx=0x7f06d4005030 comp rx=0 tx=0).stop 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 shutdown_connections 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f06c4039650 0x7f06c403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 --2- 172.21.15.186:0/1590799670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06d8108560 0x7f06d8197f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 >> 172.21.15.186:0/1590799670 conn(0x7f06d80fdfa0 msgr2=0x7f06d80fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 shutdown_connections 2024-04-20T21:23:41.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:41.637+0000 7f06e0369700 1 -- 172.21.15.186:0/1590799670 wait complete. 2024-04-20T21:23:41.638 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 1 2024-04-20T21:23:42.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:41 smithi007 ceph-mon[16867]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 systemd[1]: Starting Ceph mon.c for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:23:42.602 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 podman[22138]: 2024-04-20 21:23:42.394355739 +0000 UTC m=+0.018090397 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:23:42.602 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 podman[22138]: 2024-04-20 21:23:42.499011974 +0000 UTC m=+0.122746632 container create 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-20T21:23:42.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 podman[22138]: 2024-04-20 21:23:42.692896241 +0000 UTC m=+0.316630901 container init 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-20T21:23:42.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 podman[22138]: 2024-04-20 21:23:42.695696115 +0000 UTC m=+0.319430782 container start 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:23:42.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pidfile_write: ignore empty --pid-file 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: load: jerasure load: lrc 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: RocksDB version: 7.9.2 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Git sha 0 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: DB SUMMARY 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: DB Session ID: 5UWCQW9YRDHKLFJJ9T5B 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: CURRENT file: CURRENT 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-20T21:23:42.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.error_if_exists: 0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.create_if_missing: 0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.paranoid_checks: 1 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.env: 0x561ecfa6ccc0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.info_log: 0x561ed0545c20 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.statistics: (nil) 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.use_fsync: 0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_log_file_size: 0 2024-04-20T21:23:42.968 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_fallocate: 1 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.use_direct_reads: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.db_log_dir: 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.wal_dir: 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T21:23:42.969 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.write_buffer_manager: 0x561ed05141e0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T21:23:42.970 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.unordered_write: 0 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.row_cache: None 2024-04-20T21:23:42.971 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.wal_filter: None 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.two_write_queues: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.wal_compression: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.atomic_flush: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T21:23:42.973 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.log_readahead_size: 0 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_background_jobs: 2 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_background_compactions: -1 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_subcompactions: 1 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T21:23:42.974 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_open_files: -1 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_background_flushes: -1 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Compression algorithms supported: 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T21:23:42.975 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kZSTD supported: 0 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kXpressCompression supported: 0 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kZlibCompression supported: 1 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kSnappyCompression supported: 1 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kLZ4Compression supported: 1 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: kBZip2Compression supported: 0 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.merge_operator: 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_filter: None 2024-04-20T21:23:42.976 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561ed0545de0) 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks: 1 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: pin_top_level_index_and_filter: 1 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_type: 0 2024-04-20T21:23:42.977 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_index_type: 0 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_shortening: 1 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: checksum: 4 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: no_block_cache: 0 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache: 0x561ed0547090 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_name: BinnedLRUCache 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_options: 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: capacity : 536870912 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_shard_bits : 4 2024-04-20T21:23:42.978 INFO:journalctl@ceph.mon.c.smithi186.stdout: strict_capacity_limit : 0 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: high_pri_pool_ratio: 0.000 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_cache_compressed: (nil) 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: persistent_cache: (nil) 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size: 4096 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_size_deviation: 10 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_restart_interval: 16 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: index_block_restart_interval: 1 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: metadata_block_size: 4096 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: partition_filters: 0 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: use_delta_encoding: 1 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: filter_policy: bloomfilter 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: whole_key_filtering: 1 2024-04-20T21:23:42.979 INFO:journalctl@ceph.mon.c.smithi186.stdout: verify_compression: 0 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: read_amp_bytes_per_bit: 0 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: format_version: 5 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_index_compression: 1 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: block_align: 0 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: max_auto_readahead_size: 262144 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: prepopulate_block_cache: 0 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: initial_auto_readahead_size: 8192 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression: NoCompression 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T21:23:42.980 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.num_levels: 7 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:23:42.981 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T21:23:42.982 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T21:23:42.983 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T21:23:42.984 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T21:23:42.985 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.inplace_update_support: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.bloom_locality: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.max_successive_merges: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.ttl: 2592000 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T21:23:42.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enable_blob_files: false 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.min_blob_size: 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2d5c72cd-7bc1-449e-8196-ad35ed6686a6 2024-04-20T21:23:42.987 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648222727358, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648222727800, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713648222, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2d5c72cd-7bc1-449e-8196-ad35ed6686a6", "db_session_id": "5UWCQW9YRDHKLFJJ9T5B", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648222727915, "job": 1, "event": "recovery_finished"} 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561ed0634000 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: DB pointer 0x561ed061e000 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** DB Stats ** 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:42.988 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.989 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** Compaction Stats [default] ** 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T21:23:42.990 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache BinnedLRUCache@0x561ed0547090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: starting mon.c rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:42.991 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(???) e0 preinit fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).mds e1 new map 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: e1 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: legacy client fscid: -1 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout: No filesystems configured 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-20T21:23:42.992 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Added host smithi016 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.993 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.994 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Deploying cephadm binary to smithi186 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Added host smithi186 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.995 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:42.996 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c;count:3 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: Deploying daemon mon.c on smithi186 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.997 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:42.998 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/1590799670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:42.998 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 ceph-mon[22157]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-20T21:23:42.998 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 bash[22138]: 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 2024-04-20T21:23:42.998 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:42 smithi186 systemd[1]: Started Ceph mon.c for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:23:43.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:42 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/1590799670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:43.282 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:23:43.282 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:23:43.516 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:23:47.957 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 -- 172.21.15.186:0/319645836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc00035c0 msgr2=0x7f0cc0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:47.957 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 --2- 172.21.15.186:0/319645836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc00035c0 0x7f0cc0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0cd41024a0 tx=0x7f0ccc00feb0 comp rx=0 tx=0).stop 2024-04-20T21:23:47.957 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 -- 172.21.15.186:0/319645836 shutdown_connections 2024-04-20T21:23:47.957 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 --2- 172.21.15.186:0/319645836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc00035c0 0x7f0cc0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:47.958 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 -- 172.21.15.186:0/319645836 >> 172.21.15.186:0/319645836 conn(0x7f0cd40fe060 msgr2=0x7f0cd4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:47.958 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 -- 172.21.15.186:0/319645836 shutdown_connections 2024-04-20T21:23:47.958 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.957+0000 7f0cdb809700 1 -- 172.21.15.186:0/319645836 wait complete. 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.958+0000 7f0cdb809700 1 Processor -- start 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.958+0000 7f0cdb809700 1 -- start start 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cdb809700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cdb809700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 0x7f0cd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cdb809700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410a610 con 0x7f0cd4075290 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cdb809700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd410a490 con 0x7f0cc00035c0 2024-04-20T21:23:47.959 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:40048/0 (socket says 172.21.15.186:40048) 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 -- 172.21.15.186:0/2311146737 learned_addr learned my addr 172.21.15.186:0/2311146737 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 -- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 msgr2=0x7f0cd4074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 -- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 msgr2=0x7f0cd4074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd95a5700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.959+0000 7f0cd8da4700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 0x7f0cd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.960+0000 7f0cd8da4700 1 -- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 msgr2=0x7f0cd4074d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.960+0000 7f0cd8da4700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:47.960 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.960+0000 7f0cd8da4700 1 -- 172.21.15.186:0/2311146737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd4076c70 con 0x7f0cd4075290 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.960+0000 7f0cd8da4700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 0x7f0cd4075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400b4d0 tx=0x7f0cc400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.961+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0cc4011630 con 0x7f0cd4075290 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.961+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd41a43b0 con 0x7f0cd4075290 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.961+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4011cb0 con 0x7f0cd4075290 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.961+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd41a48f0 con 0x7f0cd4075290 2024-04-20T21:23:47.961 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.961+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0cc4010e60 con 0x7f0cd4075290 2024-04-20T21:23:47.962 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.962+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd4103cb0 con 0x7f0cd4075290 2024-04-20T21:23:47.962 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.963+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f0cc4018020 con 0x7f0cd4075290 2024-04-20T21:23:47.962 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.963+0000 7f0cca7fc700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 0x7f0cb803b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:47.962 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.963+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0cc404c470 con 0x7f0cd4075290 2024-04-20T21:23:47.963 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.963+0000 7f0cd95a5700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 0x7f0cb803b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:47.963 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.964+0000 7f0cd95a5700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 0x7f0cb803b850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00fa60 tx=0x7f0ccc005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:47.976 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:47.977+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0cc401f020 con 0x7f0cd4075290 2024-04-20T21:23:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: mon.a calling monitor election 2024-04-20T21:23:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: mon.c calling monitor election 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: monmap epoch 2 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: last_changed 2024-04-20T21:23:42.766577+0000 2024-04-20T21:23:48.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: min_mon_release 19 (squid) 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: election_strategy: 1 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: fsmap 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: mgrmap e13: x(active, since 59s) 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: overall HEALTH_OK 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:48.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:48.259 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.260+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0cd402df90 con 0x7f0cd4075290 2024-04-20T21:23:48.261 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.262+0000 7f0cca7fc700 1 -- 172.21.15.186:0/2311146737 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f0cc4019570 con 0x7f0cd4075290 2024-04-20T21:23:48.261 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:23:48.261 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":2,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:23:42.766577Z","created":"2024-04-20T21:21:28.997163Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 msgr2=0x7f0cb803b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 0x7f0cb803b850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00fa60 tx=0x7f0ccc005c20 comp rx=0 tx=0).stop 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 msgr2=0x7f0cd4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 0x7f0cd4075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400b4d0 tx=0x7f0cc400b890 comp rx=0 tx=0).stop 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 shutdown_connections 2024-04-20T21:23:48.263 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0cb8039390 0x7f0cb803b850 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cc00035c0 0x7f0cd4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 --2- 172.21.15.186:0/2311146737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd4075290 0x7f0cd4075710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 >> 172.21.15.186:0/2311146737 conn(0x7f0cd40fe060 msgr2=0x7f0cd40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 shutdown_connections 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:48.264+0000 7f0cdb809700 1 -- 172.21.15.186:0/2311146737 wait complete. 2024-04-20T21:23:48.264 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 2 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: mon.a calling monitor election 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: mon.c calling monitor election 2024-04-20T21:23:48.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: monmap epoch 2 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: last_changed 2024-04-20T21:23:42.766577+0000 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: min_mon_release 19 (squid) 2024-04-20T21:23:48.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: election_strategy: 1 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: fsmap 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: mgrmap e13: x(active, since 59s) 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: overall HEALTH_OK 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:48.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: Deploying daemon mon.b on smithi016 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/2311146737' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:48.993 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:49.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: Deploying daemon mon.b on smithi016 2024-04-20T21:23:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/2311146737' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:48 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:49.815 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-20T21:23:49.815 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-20T21:23:50.058 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:23:50.180 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:49 smithi186 ceph-mon[22157]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:50.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:49 smithi007 ceph-mon[16867]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:51.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 podman[21391]: 2024-04-20 21:23:51.351777204 +0000 UTC m=+0.018731629 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:23:51.480 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 podman[21391]: 2024-04-20 21:23:51.481305679 +0000 UTC m=+0.148260084 container create 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 -- 172.21.15.186:0/3370768951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 msgr2=0x7fb1a8103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 --2- 172.21.15.186:0/3370768951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a8103a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb198009910 tx=0x7fb198009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 -- 172.21.15.186:0/3370768951 shutdown_connections 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 --2- 172.21.15.186:0/3370768951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a8103a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 -- 172.21.15.186:0/3370768951 >> 172.21.15.186:0/3370768951 conn(0x7fb1a80fcfb0 msgr2=0x7fb1a80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 -- 172.21.15.186:0/3370768951 shutdown_connections 2024-04-20T21:23:51.566 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.563+0000 7fb1ace70700 1 -- 172.21.15.186:0/3370768951 wait complete. 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 Processor -- start 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 -- start start 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a810cc80 con 0x7fb1a81036b0 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1ace70700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a810cb00 con 0x7fb1a81986f0 2024-04-20T21:23:51.567 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1a5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1a5d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:44188/0 (socket says 172.21.15.186:44188) 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.564+0000 7fb1a5d9b700 1 -- 172.21.15.186:0/1398684511 learned_addr learned my addr 172.21.15.186:0/1398684511 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a5d9b700 1 -- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 msgr2=0x7fb1a819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a5d9b700 1 -- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 msgr2=0x7fb1a819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a5d9b700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:23:51.568 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a5d9b700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a659c700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a659c700 1 -- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 msgr2=0x7fb1a819ca80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a659c700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.565+0000 7fb1a659c700 1 -- 172.21.15.186:0/1398684511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a8198d30 con 0x7fb1a81036b0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1a659c700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a81981b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb198014000 tx=0x7fb198005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb198005dc0 con 0x7fb1a81036b0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a819cfc0 con 0x7fb1a81036b0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a819d540 con 0x7fb1a81036b0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb19800be40 con 0x7fb1a81036b0 2024-04-20T21:23:51.569 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.566+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb19800f910 con 0x7fb1a81036b0 2024-04-20T21:23:51.570 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.567+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188005330 con 0x7fb1a81036b0 2024-04-20T21:23:51.570 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.568+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fb19802a070 con 0x7fb1a81036b0 2024-04-20T21:23:51.570 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.568+0000 7fb1977fe700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 0x7fb19003b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:51.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.568+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb19801e030 con 0x7fb1a81036b0 2024-04-20T21:23:51.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.568+0000 7fb1a5d9b700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 0x7fb19003b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:51.571 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.569+0000 7fb1a5d9b700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 0x7fb19003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009180 tx=0x7fb19c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:51.582 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.583+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb198013070 con 0x7fb1a81036b0 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 podman[21391]: 2024-04-20 21:23:51.692966759 +0000 UTC m=+0.359921168 container init 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 podman[21391]: 2024-04-20 21:23:51.695951475 +0000 UTC m=+0.362905884 container start 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: set uid:gid to 167:167 (ceph:ceph) 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev), process ceph-mon, pid 2 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pidfile_write: ignore empty --pid-file 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: load: jerasure load: lrc 2024-04-20T21:23:51.730 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: RocksDB version: 7.9.2 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Git sha 0 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Compile date 2024-04-17 00:07:32 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: DB SUMMARY 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: DB Session ID: YSAKYFGM5SJ6F1ZGYNHC 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: CURRENT file: CURRENT 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: IDENTITY file: IDENTITY 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.error_if_exists: 0 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.create_if_missing: 0 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.paranoid_checks: 1 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-20T21:23:51.731 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.env: 0x56505cefacc0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.fs: PosixFileSystem 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.info_log: 0x56505e1c3c20 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_file_opening_threads: 16 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.statistics: (nil) 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.use_fsync: 0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_log_file_size: 0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.keep_log_file_num: 1000 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.recycle_log_file_num: 0 2024-04-20T21:23:51.732 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_fallocate: 1 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_mmap_reads: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_mmap_writes: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.use_direct_reads: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.create_missing_column_families: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.db_log_dir: 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.wal_dir: 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.advise_random_on_open: 1 2024-04-20T21:23:51.733 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.db_write_buffer_size: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.write_buffer_manager: 0x56505e1921e0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.rate_limiter: (nil) 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.wal_recovery_mode: 2 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enable_thread_tracking: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enable_pipelined_write: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.unordered_write: 0 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-20T21:23:51.734 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-20T21:23:51.735 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-20T21:23:51.735 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.row_cache: None 2024-04-20T21:23:51.735 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.wal_filter: None 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_ingest_behind: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.two_write_queues: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.manual_wal_flush: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.wal_compression: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.atomic_flush: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-20T21:23:51.736 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.log_readahead_size: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.best_efforts_recovery: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.allow_data_in_errors: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.db_host_id: __hostname__ 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_background_jobs: 2 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_background_compactions: -1 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_subcompactions: 1 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-20T21:23:51.737 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_total_wal_size: 0 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_open_files: -1 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bytes_per_sync: 0 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_readahead_size: 0 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_background_flushes: -1 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Compression algorithms supported: 2024-04-20T21:23:51.738 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kZSTD supported: 0 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kXpressCompression supported: 0 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kLZ4HCCompression supported: 1 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kZlibCompression supported: 1 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kSnappyCompression supported: 1 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kLZ4Compression supported: 1 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: kBZip2Compression supported: 0 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-20T21:23:51.739 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.merge_operator: 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_filter: None 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_filter_factory: None 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.sst_partitioner_factory: None 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56505e1c3de0) 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_type: 0 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: data_block_index_type: 0 2024-04-20T21:23:51.740 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_shortening: 1 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: checksum: 4 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: no_block_cache: 0 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache: 0x56505e1c5090 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_options: 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: capacity : 536870912 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: num_shard_bits : 4 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: strict_capacity_limit : 0 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_cache_compressed: (nil) 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: persistent_cache: (nil) 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_size: 4096 2024-04-20T21:23:51.741 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_size_deviation: 10 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_restart_interval: 16 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: index_block_restart_interval: 1 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: metadata_block_size: 4096 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: partition_filters: 0 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: use_delta_encoding: 1 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: filter_policy: bloomfilter 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: whole_key_filtering: 1 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: verify_compression: 0 2024-04-20T21:23:51.742 INFO:journalctl@ceph.mon.b.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: format_version: 5 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: enable_index_compression: 1 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: block_align: 0 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: max_auto_readahead_size: 262144 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: prepopulate_block_cache: 0 2024-04-20T21:23:51.743 INFO:journalctl@ceph.mon.b.smithi016.stdout: initial_auto_readahead_size: 8192 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.write_buffer_size: 33554432 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_write_buffer_number: 2 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression: NoCompression 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression: Disabled 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.prefix_extractor: nullptr 2024-04-20T21:23:51.744 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.num_levels: 7 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:23:51.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.level: 32767 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.strategy: 0 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.enabled: false 2024-04-20T21:23:51.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.target_file_size_base: 67108864 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-20T21:23:51.747 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.arena_block_size: 1048576 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.disable_auto_compactions: 0 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-20T21:23:51.748 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-20T21:23:51.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-20T21:23:51.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-20T21:23:51.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-20T21:23:51.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-20T21:23:51.749 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.inplace_update_support: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.bloom_locality: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.max_successive_merges: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.paranoid_file_checks: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.force_consistency_checks: 1 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.report_bg_io_stats: 0 2024-04-20T21:23:51.750 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.ttl: 2592000 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enable_blob_files: false 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.min_blob_size: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_file_size: 268435456 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.blob_file_starting_level: 0 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-20T21:23:51.751 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cbaddd00-41ae-424b-8ca3-547744d0bd92 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648231726369, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648231726959, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713648231, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cbaddd00-41ae-424b-8ca3-547744d0bd92", "db_session_id": "YSAKYFGM5SJ6F1ZGYNHC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713648231727071, "job": 1, "event": "recovery_finished"} 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56505e2b2000 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: DB pointer 0x56505e29c000 2024-04-20T21:23:51.752 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** DB Stats ** 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:23:51.753 INFO:journalctl@ceph.mon.b.smithi016.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** Compaction Stats [default] ** 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-20T21:23:51.754 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache BinnedLRUCache@0x56505e1c5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: using public_addr v2:172.21.15.16:0/0 -> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: starting mon.b rank -1 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:51.755 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(???) e0 preinit fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:51.756 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.755+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19800e760 con 0x7fb1a81036b0 2024-04-20T21:23:51.867 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:51.868+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb188005620 con 0x7fb1a81036b0 2024-04-20T21:23:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).mds e1 new map 2024-04-20T21:23:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-20T21:23:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout: e1 2024-04-20T21:23:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout: legacy client fscid: -1 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout: 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout: No filesystems configured 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:52.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Added host smithi016 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:52.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:52.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:52.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:52.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Deploying cephadm binary to smithi186 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.199 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Added host smithi186 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.200 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/467024116' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.201 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Saving service mon spec with placement smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c;count:3 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Deploying daemon mon.c on smithi186 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.202 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/1590799670' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.a calling monitor election 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.c calling monitor election 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.203 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: monmap epoch 2 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: last_changed 2024-04-20T21:23:42.766577+0000 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: min_mon_release 19 (squid) 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: election_strategy: 1 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: fsmap 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mgrmap e13: x(active, since 59s) 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: overall HEALTH_OK 2024-04-20T21:23:52.204 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: Deploying daemon mon.b on smithi016 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/2311146737' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:52.205 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 ceph-mon[21409]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-20T21:23:52.206 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 bash[21391]: 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 2024-04-20T21:23:52.206 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:51 smithi016 systemd[1]: Started Ceph mon.b for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:23:56.775 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.776+0000 7fb1977fe700 1 -- 172.21.15.186:0/1398684511 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7fb19802b3b0 con 0x7fb1a81036b0 2024-04-20T21:23:56.775 INFO:teuthology.orchestra.run.smithi186.stdout: 2024-04-20T21:23:56.776 INFO:teuthology.orchestra.run.smithi186.stdout:{"epoch":3,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","modified":"2024-04-20T21:23:51.753259Z","created":"2024-04-20T21:21:28.997163Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:3300","nonce":0},{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.778+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 msgr2=0x7fb19003b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.778+0000 7fb1ace70700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 0x7fb19003b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009180 tx=0x7fb19c008040 comp rx=0 tx=0).stop 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.778+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 msgr2=0x7fb1a81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.778+0000 7fb1ace70700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a81981b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb198014000 tx=0x7fb198005030 comp rx=0 tx=0).stop 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 shutdown_connections 2024-04-20T21:23:56.778 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1a81036b0 0x7fb1a81981b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb1900393f0 0x7fb19003b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 --2- 172.21.15.186:0/1398684511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb1a81986f0 0x7fb1a819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 >> 172.21.15.186:0/1398684511 conn(0x7fb1a80fcfb0 msgr2=0x7fb1a81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 shutdown_connections 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:23:56.779+0000 7fb1ace70700 1 -- 172.21.15.186:0/1398684511 wait complete. 2024-04-20T21:23:56.779 INFO:teuthology.orchestra.run.smithi186.stderr:dumped monmap epoch 3 2024-04-20T21:23:57.076 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: mon.c calling monitor election 2024-04-20T21:23:57.076 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:57.076 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: mon.a calling monitor election 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/1398684511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.077 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: monmap epoch 3 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: min_mon_release 19 (squid) 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: election_strategy: 1 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: fsmap 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:57.078 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: mgrmap e13: x(active, since 68s) 2024-04-20T21:23:57.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: overall HEALTH_OK 2024-04-20T21:23:57.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.079 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: mon.c calling monitor election 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: mon.a calling monitor election 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/1398684511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:57.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:57.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:57.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: monmap epoch 3 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:57.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: min_mon_release 19 (squid) 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: election_strategy: 1 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:57.287 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: fsmap 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: mgrmap e13: x(active, since 68s) 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: overall HEALTH_OK 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.288 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.289 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:57.379 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-20T21:23:57.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-20T21:23:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:23:58.142 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:57 smithi007 ceph-mon[16867]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:58.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:58.143 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:58.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:57 smithi186 ceph-mon[22157]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: mon.c calling monitor election 2024-04-20T21:23:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:23:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: mon.a calling monitor election 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/1398684511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: monmap epoch 3 2024-04-20T21:23:59.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: min_mon_release 19 (squid) 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: election_strategy: 1 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: fsmap 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: mgrmap e13: x(active, since 68s) 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: overall HEALTH_OK 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:23:59.198 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:23:59.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mon.b calling monitor election 2024-04-20T21:23:59.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:23:59.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:59.699 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mon.b calling monitor election 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mon.c calling monitor election 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mon.a calling monitor election 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: monmap epoch 3 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: min_mon_release 19 (squid) 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: election_strategy: 1 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:59.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: fsmap 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: mgrmap e13: x(active, since 70s) 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: overall HEALTH_OK 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:23:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mon.b calling monitor election 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mon.b calling monitor election 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mon.c calling monitor election 2024-04-20T21:23:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mon.a calling monitor election 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: monmap epoch 3 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: min_mon_release 19 (squid) 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: election_strategy: 1 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:59.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: fsmap 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: mgrmap e13: x(active, since 70s) 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: overall HEALTH_OK 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:23:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 -- 172.21.15.7:0/2701020399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c107390 msgr2=0x7f9c8c069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 --2- 172.21.15.7:0/2701020399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c107390 0x7f9c8c069e70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009910 tx=0x7f9c80009c20 comp rx=0 tx=0).stop 2024-04-20T21:23:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 -- 172.21.15.7:0/2701020399 shutdown_connections 2024-04-20T21:23:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 --2- 172.21.15.7:0/2701020399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c107390 0x7f9c8c069e70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 -- 172.21.15.7:0/2701020399 >> 172.21.15.7:0/2701020399 conn(0x7f9c8c076eb0 msgr2=0x7f9c8c0772c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 -- 172.21.15.7:0/2701020399 shutdown_connections 2024-04-20T21:23:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.868+0000 7f9c94600700 1 -- 172.21.15.7:0/2701020399 wait complete. 2024-04-20T21:23:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.869+0000 7f9c94600700 1 Processor -- start 2024-04-20T21:23:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 -- start start 2024-04-20T21:23:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 0x7f9c8c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c19ec70 0x7f9c8c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c113110 con 0x7f9c8c1a4b00 2024-04-20T21:23:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c112f90 con 0x7f9c8c19ec70 2024-04-20T21:23:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c94600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8c113290 con 0x7f9c8c107390 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.870+0000 7f9c91200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44036/0 (socket says 172.21.15.7:44036) 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91200700 1 -- 172.21.15.7:0/858625301 learned_addr learned my addr 172.21.15.7:0/858625301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 0x7f9c8c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c92600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c19ec70 0x7f9c8c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 -- 172.21.15.7:0/858625301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c19ec70 msgr2=0x7f9c8c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c19ec70 0x7f9c8c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 -- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 msgr2=0x7f9c8c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91c00700 1 -- 172.21.15.7:0/858625301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8c1ab370 con 0x7f9c8c107390 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.871+0000 7f9c91200700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c91c00700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 0x7f9c8c1a45c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005950 tx=0x7f9c80004f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80026070 con 0x7f9c8c107390 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c8000b750 con 0x7f9c8c107390 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c800038e0 con 0x7f9c8c107390 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8c1ab5a0 con 0x7f9c8c107390 2024-04-20T21:23:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.872+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8c1aba30 con 0x7f9c8c107390 2024-04-20T21:23:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.873+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f9c80029070 con 0x7f9c8c107390 2024-04-20T21:23:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.873+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8c101980 con 0x7f9c8c107390 2024-04-20T21:23:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.874+0000 7f9c8a000700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 0x7f9c7403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.874+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9c8004d330 con 0x7f9c8c107390 2024-04-20T21:23:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.874+0000 7f9c91200700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 0x7f9c7403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.875+0000 7f9c91200700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 0x7f9c7403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009f70 tx=0x7f9c780092c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:23:59.880+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9c80013070 con 0x7f9c8c107390 2024-04-20T21:23:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mon.b calling monitor election 2024-04-20T21:23:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mon.b calling monitor election 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mon.c calling monitor election 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mon.a calling monitor election 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: monmap epoch 3 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:23:59.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: last_changed 2024-04-20T21:23:51.753259+0000 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: created 2024-04-20T21:21:28.997163+0000 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: min_mon_release 19 (squid) 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: election_strategy: 1 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: 0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: 1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.c 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: 2: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.b 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: fsmap 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: osdmap e4: 0 total, 0 up, 0 in 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: mgrmap e13: x(active, since 70s) 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: overall HEALTH_OK 2024-04-20T21:23:59.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:23:59.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:23:59.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:23:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.134+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9c8c1abdc0 con 0x7f9c8c107390 2024-04-20T21:24:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.135+0000 7f9c8a000700 1 -- 172.21.15.7:0/858625301 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f9c80030090 con 0x7f9c8c107390 2024-04-20T21:24:00.134 INFO:teuthology.orchestra.run.smithi007.stdout:# minimal ceph.conf for 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:00.134 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-04-20T21:24:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:24:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: mon_host = [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 msgr2=0x7f9c7403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 0x7f9c7403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009f70 tx=0x7f9c780092c0 comp rx=0 tx=0).stop 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 msgr2=0x7f9c8c1a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 0x7f9c8c1a45c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c80005950 tx=0x7f9c80004f80 comp rx=0 tx=0).stop 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 shutdown_connections 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9c74039ae0 0x7f9c7403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c8c107390 0x7f9c8c1a45c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8c19ec70 0x7f9c8c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 --2- 172.21.15.7:0/858625301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c8c1a4b00 0x7f9c8c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:00.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 >> 172.21.15.7:0/858625301 conn(0x7f9c8c076eb0 msgr2=0x7f9c8c0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:00.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 shutdown_connections 2024-04-20T21:24:00.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:00.138+0000 7f9c94600700 1 -- 172.21.15.7:0/858625301 wait complete. 2024-04-20T21:24:00.753 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:00.754 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:00.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:00.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: Reconfiguring daemon mon.a on smithi007 2024-04-20T21:24:00.755 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/858625301' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:00.896 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-20T21:24:00.897 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:24:00.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:00.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:00.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:00.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: Reconfiguring daemon mon.a on smithi007 2024-04-20T21:24:00.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:00 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/858625301' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:00.964 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:24:00.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:00.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: Reconfiguring mon.a (unknown last config time)... 2024-04-20T21:24:00.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:00.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:00.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:00.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: Reconfiguring daemon mon.a on smithi007 2024-04-20T21:24:00.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:00 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/858625301' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:01.032 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:24:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:24:00.753+0000 7f519c000700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T21:24:01.072 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:24:01.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:01.105 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:24:01.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:01.178 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:24:01.178 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2024-04-20T21:24:01.209 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:24:01.209 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:24:01.276 INFO:tasks.cephadm:Adding mgr.x on smithi007 2024-04-20T21:24:01.277 INFO:tasks.cephadm:Adding mgr.y on smithi016 2024-04-20T21:24:01.277 INFO:tasks.cephadm:Adding mgr.z on smithi186 2024-04-20T21:24:01.277 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi007=x;smithi016=y;smithi186=z' 2024-04-20T21:24:01.559 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:24:02.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:02 smithi016 ceph-mon[21409]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:02.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:02 smithi186 ceph-mon[22157]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:02.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:02 smithi007 ceph-mon[16867]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- 172.21.15.186:0/3331276898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c1025c0 msgr2=0x7fea9c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 --2- 172.21.15.186:0/3331276898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c1025c0 0x7fea9c1029e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea9800b0a0 tx=0x7fea9800b3b0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- 172.21.15.186:0/3331276898 shutdown_connections 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 --2- 172.21.15.186:0/3331276898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c1025c0 0x7fea9c1029e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- 172.21.15.186:0/3331276898 >> 172.21.15.186:0/3331276898 conn(0x7fea9c0fdb80 msgr2=0x7fea9c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.005 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- 172.21.15.186:0/3331276898 shutdown_connections 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- 172.21.15.186:0/3331276898 wait complete. 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 Processor -- start 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.004+0000 7feaa367b700 1 -- start start 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c074e40 0x7fea9c0752c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c075800 0x7fea9c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c108600 con 0x7fea9c074e40 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c108480 con 0x7fea9c075800 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c108780 con 0x7fea9c1025c0 2024-04-20T21:24:03.006 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:50440/0 (socket says 172.21.15.186:50440) 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 -- 172.21.15.186:0/1073915798 learned_addr learned my addr 172.21.15.186:0/1073915798 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 -- 172.21.15.186:0/1073915798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c075800 msgr2=0x7fea9c1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c075800 0x7fea9c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 -- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c074e40 msgr2=0x7fea9c0752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c074e40 0x7fea9c0752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 -- 172.21.15.186:0/1073915798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a2410 con 0x7fea9c1025c0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa1417700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fea98000f80 tx=0x7fea98009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98017040 con 0x7fea9c1025c0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea98020070 con 0x7fea9c1025c0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1a26a0 con 0x7fea9c1025c0 2024-04-20T21:24:03.007 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98012790 con 0x7fea9c1025c0 2024-04-20T21:24:03.008 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.005+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a2c00 con 0x7fea9c1025c0 2024-04-20T21:24:03.008 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.006+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7fea98005670 con 0x7fea9c1025c0 2024-04-20T21:24:03.009 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.006+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c0640b0 con 0x7fea9c1025c0 2024-04-20T21:24:03.009 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.006+0000 7fea927fc700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 0x7fea8803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.009 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.007+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fea9800f020 con 0x7fea9c1025c0 2024-04-20T21:24:03.009 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.007+0000 7feaa0c16700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 0x7fea8803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.009 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.008+0000 7feaa0c16700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 0x7fea8803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0098e0 tx=0x7fea8c006dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.011 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.012+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fea980184e0 con 0x7fea9c1025c0 2024-04-20T21:24:03.224 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.225+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi186=z", "target": ["mon-mgr", ""]}) -- 0x7fea9c062f20 con 0x7fea88039ae0 2024-04-20T21:24:03.238 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.239+0000 7fea927fc700 1 -- 172.21.15.186:0/1073915798 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fea9c062f20 con 0x7fea88039ae0 2024-04-20T21:24:03.238 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mgr update... 2024-04-20T21:24:03.240 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 msgr2=0x7fea8803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.240 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 0x7fea8803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0098e0 tx=0x7fea8c006dd0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.240 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 msgr2=0x7fea9c0767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.240 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fea98000f80 tx=0x7fea98009540 comp rx=0 tx=0).stop 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 shutdown_connections 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fea88039ae0 0x7fea8803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea9c1025c0 0x7fea9c0767f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c075800 0x7fea9c1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 --2- 172.21.15.186:0/1073915798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea9c074e40 0x7fea9c0752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 >> 172.21.15.186:0/1073915798 conn(0x7fea9c0fdb80 msgr2=0x7fea9c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 shutdown_connections 2024-04-20T21:24:03.241 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:24:03.241+0000 7feaa367b700 1 -- 172.21.15.186:0/1073915798 wait complete. 2024-04-20T21:24:03.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:03.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: Reconfiguring daemon mon.b on smithi016 2024-04-20T21:24:03.685 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: Reconfiguring daemon mon.b on smithi016 2024-04-20T21:24:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: Reconfiguring mon.b (monmap changed)... 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:03.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: Reconfiguring daemon mon.b on smithi016 2024-04-20T21:24:03.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:03.828 DEBUG:teuthology.orchestra.run.smithi016:mgr.y> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.y.service 2024-04-20T21:24:03.831 DEBUG:teuthology.orchestra.run.smithi186:mgr.z> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.z.service 2024-04-20T21:24:03.833 INFO:tasks.cephadm:Deploying OSDs... 2024-04-20T21:24:03.833 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:24:03.833 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:03.891 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:03.892 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:03.944 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-20 21:23:02.080988861 +0000 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-20 21:14:12.961370319 +0000 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-20 21:14:12.961370319 +0000 2024-04-20T21:24:03.945 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-20 21:14:12.961370319 +0000 2024-04-20T21:24:03.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-20T21:24:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-20T21:24:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000293731 s, 1.7 MB/s 2024-04-20T21:24:04.041 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:04.100 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:04.150 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:04.150 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:04.150 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-20 21:23:02.190987208 +0000 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-20 21:14:13.577360912 +0000 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-20 21:14:13.577360912 +0000 2024-04-20T21:24:04.151 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-20 21:14:13.577360912 +0000 2024-04-20T21:24:04.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-20T21:24:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-20T21:24:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000280156 s, 1.8 MB/s 2024-04-20T21:24:04.232 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:04.289 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-20 21:23:02.304985494 +0000 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-20 21:14:14.194351489 +0000 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-20 21:14:14.194351489 +0000 2024-04-20T21:24:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-20 21:14:14.194351489 +0000 2024-04-20T21:24:04.341 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:04.417 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-20T21:24:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-20T21:24:04.418 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000271882 s, 1.9 MB/s 2024-04-20T21:24:04.422 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:04.480 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-20 21:23:02.402984022 +0000 2024-04-20T21:24:04.533 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-20 21:14:14.808342112 +0000 2024-04-20T21:24:04.534 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-20 21:14:14.808342112 +0000 2024-04-20T21:24:04.534 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-20 21:14:14.808342112 +0000 2024-04-20T21:24:04.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-20T21:24:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-20T21:24:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000239618 s, 2.1 MB/s 2024-04-20T21:24:04.617 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:04.675 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-04-20T21:24:04.675 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:04.695 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:04.695 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:04.753 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:04.753 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-20 21:23:51.143380125 +0000 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-20 21:13:59.112104913 +0000 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-20 21:13:59.112104913 +0000 2024-04-20T21:24:04.754 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-20 21:13:59.112104913 +0000 2024-04-20T21:24:04.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:04.822 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-20T21:24:04.822 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-20T21:24:04.822 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000130093 s, 3.9 MB/s 2024-04-20T21:24:04.823 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:04.883 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-20 21:23:51.144380112 +0000 2024-04-20T21:24:04.941 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-20 21:13:59.459100301 +0000 2024-04-20T21:24:04.942 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-20 21:13:59.459100301 +0000 2024-04-20T21:24:04.942 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-20 21:13:59.459100301 +0000 2024-04-20T21:24:04.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:05.007 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-20T21:24:05.007 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-20T21:24:05.007 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000120965 s, 4.2 MB/s 2024-04-20T21:24:05.008 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:05.066 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-20 21:23:51.144380112 +0000 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-20 21:13:59.809095650 +0000 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-20 21:13:59.809095650 +0000 2024-04-20T21:24:05.123 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-20 21:13:59.809095650 +0000 2024-04-20T21:24:05.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:05.185 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi186=z;count:3 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:05.186 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:05.187 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:05.187 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:04 smithi016 ceph-mon[21409]: Reconfiguring daemon mon.c on smithi186 2024-04-20T21:24:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-20T21:24:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-20T21:24:05.188 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000105792 s, 4.8 MB/s 2024-04-20T21:24:05.190 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi186=z;count:3 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:05.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:04 smithi186 ceph-mon[22157]: Reconfiguring daemon mon.c on smithi186 2024-04-20T21:24:05.247 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi007=x;smithi016=y;smithi186=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: Saving service mgr spec with placement smithi007=x;smithi016=y;smithi186=z;count:3 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: Reconfiguring mon.c (monmap changed)... 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-20T21:24:05.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-20T21:24:05.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:05.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:04 smithi007 ceph-mon[16867]: Reconfiguring daemon mon.c on smithi186 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-04-20 21:23:51.144380112 +0000 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-04-20 21:14:00.140091252 +0000 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-04-20 21:14:00.140091252 +0000 2024-04-20T21:24:05.305 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-04-20 21:14:00.140091252 +0000 2024-04-20T21:24:05.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:05.370 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-04-20T21:24:05.370 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-04-20T21:24:05.370 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000119745 s, 4.3 MB/s 2024-04-20T21:24:05.371 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:05.429 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2024-04-20T21:24:05.429 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:24:05.445 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:24:05.446 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2024-04-20T21:24:05.501 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-20 21:23:42.024290786 +0000 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-20 21:14:06.511469914 +0000 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-20 21:14:06.511469914 +0000 2024-04-20T21:24:05.502 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-20 21:14:06.511469914 +0000 2024-04-20T21:24:05.503 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:24:05.565 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-20T21:24:05.615 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-20T21:24:05.615 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000269671 s, 1.9 MB/s 2024-04-20T21:24:05.616 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:24:05.673 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-20 21:23:42.024290786 +0000 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-20 21:14:06.860462931 +0000 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-20 21:14:06.860462931 +0000 2024-04-20T21:24:05.728 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-20 21:14:06.860462931 +0000 2024-04-20T21:24:05.729 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:24:05.790 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-20T21:24:05.790 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-20T21:24:05.790 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000189045 s, 2.7 MB/s 2024-04-20T21:24:05.792 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:24:05.850 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-20 21:23:42.024290786 +0000 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-20 21:14:07.226455608 +0000 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-20 21:14:07.226455608 +0000 2024-04-20T21:24:05.902 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-20 21:14:07.226455608 +0000 2024-04-20T21:24:05.903 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:24:05.966 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-20T21:24:05.966 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-20T21:24:05.966 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000194169 s, 2.6 MB/s 2024-04-20T21:24:05.967 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:24:06.024 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:24:06.079 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2024-04-20 21:23:42.025290766 +0000 2024-04-20T21:24:06.080 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2024-04-20 21:14:07.625447624 +0000 2024-04-20T21:24:06.080 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2024-04-20 21:14:07.625447624 +0000 2024-04-20T21:24:06.080 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: 2024-04-20 21:14:07.625447624 +0000 2024-04-20T21:24:06.080 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:24:06.103 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:06.103 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.103 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.103 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:06.104 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.125 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2024-04-20T21:24:06.125 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2024-04-20T21:24:06.126 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000199701 s, 2.6 MB/s 2024-04-20T21:24:06.127 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:24:06.183 INFO:tasks.cephadm:Deploying osd.0 on smithi007 with /dev/vg_nvme/lv_4... 2024-04-20T21:24:06.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:24:06.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:06.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.265 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:06.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:07.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:06 smithi007 ceph-mon[16867]: Deploying daemon mgr.z on smithi186 2024-04-20T21:24:07.148 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:06 smithi186 ceph-mon[22157]: Deploying daemon mgr.z on smithi186 2024-04-20T21:24:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:06 smithi016 ceph-mon[21409]: Deploying daemon mgr.z on smithi186 2024-04-20T21:24:07.466 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 systemd[1]: Starting Ceph mgr.z for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:24:07.803 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 podman[24003]: 2024-04-20 21:24:07.434223825 +0000 UTC m=+0.017581570 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:24:07.803 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 podman[24003]: 2024-04-20 21:24:07.533283349 +0000 UTC m=+0.116641090 container create 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:24:07.803 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 podman[24003]: 2024-04-20 21:24:07.777299051 +0000 UTC m=+0.360656782 container init 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:24:07.803 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 podman[24003]: 2024-04-20 21:24:07.779817509 +0000 UTC m=+0.363175249 container start 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:24:08.116 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:07 smithi186 ceph-mon[22157]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:08.116 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 ceph-mgr[24022]: -- 172.21.15.186:0/3492557312 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b69a96c5a0 con 0x55b69a940800 2024-04-20T21:24:08.116 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 bash[24003]: 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e 2024-04-20T21:24:08.116 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 systemd[1]: Started Ceph mgr.z for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:24:08.117 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:07 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:07.930+0000 7eff2316a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:24:08.117 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:08.117+0000 7eff2316a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:08.127 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:07 smithi007 ceph-mon[16867]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:08.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:07 smithi016 ceph-mon[21409]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:08.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:08.218+0000 7eff2316a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:24:08.950 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:08.524+0000 7eff2316a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:24:08.950 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:08.607+0000 7eff2316a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:24:09.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:09.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:08 smithi186 ceph-mon[22157]: Deploying daemon mgr.y on smithi016 2024-04-20T21:24:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:08 smithi007 ceph-mon[16867]: Deploying daemon mgr.y on smithi016 2024-04-20T21:24:09.338 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:09.339 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:08 smithi016 ceph-mon[21409]: Deploying daemon mgr.y on smithi016 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374495 s, 280 MB/s 2024-04-20T21:24:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-04-20T21:24:10.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:09 smithi007 ceph-mon[16867]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:10.136 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:09 smithi186 ceph-mon[22157]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:10.137 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:09 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:09.796+0000 7eff2316a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:24:10.137 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:09 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:09.965+0000 7eff2316a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:10.137 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:10 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:10.137+0000 7eff2316a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:24:10.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_4 2024-04-20T21:24:10.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:09 smithi016 ceph-mon[21409]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:10.195 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 systemd[1]: Starting Ceph mgr.y for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:24:10.529 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 podman[22783]: 2024-04-20 21:24:10.173331852 +0000 UTC m=+0.017846287 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:24:10.530 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 podman[22783]: 2024-04-20 21:24:10.284863334 +0000 UTC m=+0.129377788 container create 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:24:10.530 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 podman[22783]: 2024-04-20 21:24:10.504045251 +0000 UTC m=+0.348559667 container init 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:24:10.530 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 podman[22783]: 2024-04-20 21:24:10.506753706 +0000 UTC m=+0.351268122 container start 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-20T21:24:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:24:10.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:10 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:10.421+0000 7eff2316a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:24:10.716 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:10 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:10.512+0000 7eff2316a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:24:10.852 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 ceph-mgr[22802]: -- 172.21.15.16:0/657502319 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c92372c5a0 con 0x55c923700800 2024-04-20T21:24:10.852 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 bash[22783]: 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 2024-04-20T21:24:10.853 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 systemd[1]: Started Ceph mgr.y for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:24:10.853 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:10.661+0000 7fed8eb54200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:24:10.853 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:10.853+0000 7fed8eb54200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:11.194 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:10 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:10.959+0000 7fed8eb54200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:24:11.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:10 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:10.716+0000 7eff2316a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:24:11.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:10 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:10.907+0000 7eff2316a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:11.656 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:11.289+0000 7fed8eb54200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:24:11.656 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:11.378+0000 7fed8eb54200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.822 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:11 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:11.856 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:11.540+0000 7eff2316a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:24:11.856 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:11.857 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.857 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.857 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.857 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.857 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:11.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:11.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:11.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:11 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:12.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:11.976+0000 7eff2316a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:24:12.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:12.073+0000 7eff2316a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:24:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 -- 172.21.15.7:0/1569006143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 msgr2=0x7f4ab4111430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 --2- 172.21.15.7:0/1569006143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab4111430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009990 tx=0x7f4aac009ca0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 -- 172.21.15.7:0/1569006143 shutdown_connections 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 --2- 172.21.15.7:0/1569006143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab4111430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 --2- 172.21.15.7:0/1569006143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab4108fe0 0x7f4ab4109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 --2- 172.21.15.7:0/1569006143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab410a1e0 0x7f4ab410a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 -- 172.21.15.7:0/1569006143 >> 172.21.15.7:0/1569006143 conn(0x7f4ab4077fb0 msgr2=0x7f4ab407a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 -- 172.21.15.7:0/1569006143 shutdown_connections 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.649+0000 7f4ab9400700 1 -- 172.21.15.7:0/1569006143 wait complete. 2024-04-20T21:24:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.651+0000 7f4ab9400700 1 Processor -- start 2024-04-20T21:24:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.651+0000 7f4ab9400700 1 -- start start 2024-04-20T21:24:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.651+0000 7f4ab9400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 0x7f4ab419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab9400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab9400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab41a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab9400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4114c00 con 0x7f4ab4108fe0 2024-04-20T21:24:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab9400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4114a80 con 0x7f4ab410a1e0 2024-04-20T21:24:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab9400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4114d80 con 0x7f4ab410ab80 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 0x7f4ab419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 0x7f4ab419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37982/0 (socket says 172.21.15.7:37982) 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab3e00700 1 -- 172.21.15.7:0/2785982870 learned_addr learned my addr 172.21.15.7:0/2785982870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33078/0 (socket says 172.21.15.7:33078) 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4ab8a00700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab41a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 -- 172.21.15.7:0/2785982870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 msgr2=0x7f4ab41a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab41a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 -- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 msgr2=0x7f4ab419f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 0x7f4ab419f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.652+0000 7f4aab400700 1 -- 172.21.15.7:0/2785982870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab41a6dc0 con 0x7f4ab410a1e0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4aab400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00e9f0 tx=0x7f4a9c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00cb70 con 0x7f4ab410a1e0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a9c00ee90 con 0x7f4ab410a1e0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab41a70b0 con 0x7f4ab410a1e0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c003df0 con 0x7f4ab410a1e0 2024-04-20T21:24:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.653+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41a7640 con 0x7f4ab410a1e0 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.654+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46613+0+0 (secure 0 0 0) 0x7f4a9c010430 con 0x7f4ab410a1e0 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.654+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab4069740 con 0x7f4ab410a1e0 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.655+0000 7f4ab1600700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 0x7f4a9803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.655+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4a9c04cd20 con 0x7f4ab410a1e0 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.655+0000 7f4ab3e00700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 0x7f4a9803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.656+0000 7f4ab3e00700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 0x7f4a9803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400f5e0 tx=0x7f4aa400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:12.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.660+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4a9c01f330 con 0x7f4ab410a1e0 2024-04-20T21:24:12.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:12.276+0000 7eff2316a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:24:12.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:12.917+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4ab4062f50 con 0x7f4a98039890 2024-04-20T21:24:12.944 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:12 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:12.685+0000 7fed8eb54200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:24:12.944 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:12 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:12.859+0000 7fed8eb54200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:13.139 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:12.810+0000 7eff2316a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:24:13.140 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:12.900+0000 7eff2316a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:24:13.331 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:13.035+0000 7fed8eb54200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:24:13.331 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:13.332+0000 7fed8eb54200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:24:13.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:13.140+0000 7eff2316a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:13.272+0000 7eff2316a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:13.366+0000 7eff2316a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:13.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: Reconfiguring daemon mgr.x on smithi007 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:13.605 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: Reconfiguring daemon mgr.x on smithi007 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.606 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.635 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:13.427+0000 7fed8eb54200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:13.636+0000 7fed8eb54200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:13.636 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: Reconfiguring mgr.x (unknown last config time)... 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: Reconfiguring daemon mgr.x on smithi007 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:13.637 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:13.783 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:13.482+0000 7eff2316a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:13.783 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:13.783+0000 7eff2316a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:24:13.944 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:13.835+0000 7fed8eb54200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:24:14.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:14 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:14.373+0000 7eff2316a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:24:14.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:24:14 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:24:14.585+0000 7eff2316a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:24:14.898 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:14.461+0000 7fed8eb54200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:24:15.195 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:14.898+0000 7fed8eb54200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:24:15.195 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:14.998+0000 7fed8eb54200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:24:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:15.372+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 92601+0+0 (secure 0 0 0) 0x7f4a9c00fcb0 con 0x7f4ab410a1e0 2024-04-20T21:24:15.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:15.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: Standby manager daemon z started 2024-04-20T21:24:15.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:15.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:15.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:15.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:15.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:15.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:15.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:15.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:15.205+0000 7fed8eb54200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: Standby manager daemon z started 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:15.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:15.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:15.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: Standby manager daemon z started 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/3531468218' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:15.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:16.092 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:15.748+0000 7fed8eb54200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:24:16.092 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:15.842+0000 7fed8eb54200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:24:16.378 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:16.092+0000 7fed8eb54200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:24:16.378 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:16.230+0000 7fed8eb54200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:24:16.378 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:16.327+0000 7fed8eb54200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:24:16.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:16 smithi007 ceph-mon[16867]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:16.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:16.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-mon[21409]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:16.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:16.695 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:16.447+0000 7fed8eb54200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:24:16.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:16 smithi186 ceph-mon[22157]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-20T21:24:16.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:24:17.194 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:16.761+0000 7fed8eb54200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:24:17.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:17 smithi016 ceph-mon[21409]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:17.695 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:17 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:17.385+0000 7fed8eb54200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:24:17.696 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:24:17 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:24:17.608+0000 7fed8eb54200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:24:17.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:17 smithi186 ceph-mon[22157]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:17 smithi007 ceph-mon[16867]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:18.400+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mon.2 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f4a9c00f3c0 con 0x7f4ab410a1e0 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: Standby manager daemon y started 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3521442270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]': finished 2024-04-20T21:24:18.685 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:24:18.686 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:18 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:18.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: Standby manager daemon y started 2024-04-20T21:24:18.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3521442270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]': finished 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:24:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:18 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: Standby manager daemon y started 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2749461913' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3521442270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]: dispatch 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f182508-4c5f-4296-9117-738a5d55d302"}]': finished 2024-04-20T21:24:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: osdmap e5: 1 total, 0 up, 1 in 2024-04-20T21:24:18.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:18 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:19 smithi016 ceph-mon[21409]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:19 smithi016 ceph-mon[21409]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-20T21:24:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:24:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:19 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1046576210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:19.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:19 smithi007 ceph-mon[16867]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:19.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:19 smithi007 ceph-mon[16867]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-20T21:24:19.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:24:19.711 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:19 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1046576210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:19 smithi186 ceph-mon[22157]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:19 smithi186 ceph-mon[22157]: mgrmap e15: x(active, since 90s), standbys: y, z 2024-04-20T21:24:19.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:24:19.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:19 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1046576210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:21 smithi016 ceph-mon[21409]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:21.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:21 smithi186 ceph-mon[22157]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:21 smithi007 ceph-mon[16867]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:23 smithi016 ceph-mon[21409]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:23 smithi186 ceph-mon[22157]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:23.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:23 smithi007 ceph-mon[16867]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:25.602 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:25 smithi007 ceph-mon[16867]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:25 smithi016 ceph-mon[21409]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:25.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:25 smithi186 ceph-mon[22157]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:27 smithi016 ceph-mon[21409]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:24:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:27 smithi016 ceph-mon[21409]: Deploying daemon osd.0 on smithi007 2024-04-20T21:24:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:27 smithi186 ceph-mon[22157]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:24:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:27 smithi186 ceph-mon[22157]: Deploying daemon osd.0 on smithi007 2024-04-20T21:24:27.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:27 smithi007 ceph-mon[16867]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-20T21:24:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:27 smithi007 ceph-mon[16867]: Deploying daemon osd.0 on smithi007 2024-04-20T21:24:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:29 smithi016 ceph-mon[21409]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:29.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:29 smithi186 ceph-mon[22157]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:29 smithi007 ceph-mon[16867]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:31 smithi016 ceph-mon[21409]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:31 smithi186 ceph-mon[22157]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:31 smithi007 ceph-mon[16867]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:33 smithi016 ceph-mon[21409]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:33 smithi186 ceph-mon[22157]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:33 smithi007 ceph-mon[16867]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:35.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:35 smithi016 ceph-mon[21409]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:35 smithi186 ceph-mon[22157]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:35.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:35 smithi007 ceph-mon[16867]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:36.698 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.698 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.698 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:36.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:36.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:37.757 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:37 smithi007 ceph-mon[16867]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:37.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:37 smithi016 ceph-mon[21409]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:37.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:37 smithi186 ceph-mon[22157]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:39 smithi016 ceph-mon[21409]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:39 smithi186 ceph-mon[22157]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:24:40.001 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:40.002 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:39 smithi007 ceph-mon[16867]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-20T21:24:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.116+0000 7f4ab1600700 1 -- 172.21.15.7:0/2785982870 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4ab4062f50 con 0x7f4a98039890 2024-04-20T21:24:40.115 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 0 on host 'smithi007' 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 msgr2=0x7f4a9803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 0x7f4a9803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400f5e0 tx=0x7f4aa400b040 comp rx=0 tx=0).stop 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 msgr2=0x7f4ab419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c00e9f0 tx=0x7f4a9c00ed00 comp rx=0 tx=0).stop 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 shutdown_connections 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f4a98039890 0x7f4a9803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ab410ab80 0x7f4ab41a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ab4108fe0 0x7f4ab419f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 --2- 172.21.15.7:0/2785982870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab410a1e0 0x7f4ab419f6a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.119+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 >> 172.21.15.7:0/2785982870 conn(0x7f4ab4077fb0 msgr2=0x7f4ab410d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.120+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 shutdown_connections 2024-04-20T21:24:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:24:40.121+0000 7f4ab9400700 1 -- 172.21.15.7:0/2785982870 wait complete. 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:40.909 DEBUG:teuthology.orchestra.run.smithi007:osd.0> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.0.service 2024-04-20T21:24:40.915 INFO:tasks.cephadm:Deploying osd.1 on smithi016 with /dev/vg_nvme/lv_4... 2024-04-20T21:24:40.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:41.038 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:41.171 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: osdmap e6: 1 total, 0 up, 1 in 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:41.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:24:41.502 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0[29267]: 2024-04-20T21:24:41.382+0000 7f7403600700 -1 osd.0 0 waiting for initial osdmap 2024-04-20T21:24:41.502 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0[29267]: 2024-04-20T21:24:41.400+0000 7f73f9400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:24:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:24:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:24:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:41 smithi007 ceph-mon[16867]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:41.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:41 smithi016 ceph-mon[21409]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' 2024-04-20T21:24:42.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:24:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: osdmap e7: 1 total, 0 up, 1 in 2024-04-20T21:24:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:42.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:41 smithi186 ceph-mon[22157]: from='osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387]' entity='osd.0' 2024-04-20T21:24:42.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:24:42.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:24:42.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:42.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] boot 2024-04-20T21:24:42.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:24:42.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:42 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] boot 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:24:42.955 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:42 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:24:43.079 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:24:43.080 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-20T21:24:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-20T21:24:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0098235 s, 1.1 GB/s 2024-04-20T21:24:43.080 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-20T21:24:43.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:24:43.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:24:43.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:43.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: osd.0 [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] boot 2024-04-20T21:24:43.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: osdmap e8: 1 total, 1 up, 1 in 2024-04-20T21:24:43.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:42 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:24:43.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-04-20T21:24:43.929 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:24:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:43 smithi016 ceph-mon[21409]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:43 smithi016 ceph-mon[21409]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:24:44.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:43 smithi007 ceph-mon[16867]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:44.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:43 smithi007 ceph-mon[16867]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:24:44.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:43 smithi186 ceph-mon[22157]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-20T21:24:44.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:43 smithi186 ceph-mon[22157]: osdmap e9: 1 total, 1 up, 1 in 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 -- 172.21.15.16:0/3287341761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0103cb0 msgr2=0x7f71c0104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 --2- 172.21.15.16:0/3287341761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0103cb0 0x7f71c0104110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71b00099c0 tx=0x7f71b0009cd0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 -- 172.21.15.16:0/3287341761 shutdown_connections 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 --2- 172.21.15.16:0/3287341761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0104650 0x7f71c010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 --2- 172.21.15.16:0/3287341761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0102ab0 0x7f71c0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 --2- 172.21.15.16:0/3287341761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0103cb0 0x7f71c0104110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 -- 172.21.15.16:0/3287341761 >> 172.21.15.16:0/3287341761 conn(0x7f71c00fe050 msgr2=0x7f71c0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 -- 172.21.15.16:0/3287341761 shutdown_connections 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 -- 172.21.15.16:0/3287341761 wait complete. 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.430+0000 7f71c8185700 1 Processor -- start 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 -- start start 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0102ab0 0x7f71c0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0103cb0 0x7f71c01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010e710 con 0x7f71c0103cb0 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010e590 con 0x7f71c0104650 2024-04-20T21:24:45.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c8185700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010e890 con 0x7f71c0102ab0 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35992/0 (socket says 172.21.15.16:35992) 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 -- 172.21.15.16:0/4143654944 learned_addr learned my addr 172.21.15.16:0/4143654944 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c5f21700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0102ab0 0x7f71c0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 -- 172.21.15.16:0/4143654944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0102ab0 msgr2=0x7f71c0198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0102ab0 0x7f71c0198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 -- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0103cb0 msgr2=0x7f71c01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0103cb0 0x7f71c01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 -- 172.21.15.16:0/4143654944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01a09c0 con 0x7f71c0104650 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.431+0000 7f71c6722700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71b800ada0 tx=0x7f71b800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.432+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017020 con 0x7f71c0104650 2024-04-20T21:24:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.432+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b800a0d0 con 0x7f71c0104650 2024-04-20T21:24:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.432+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01a0cb0 con 0x7f71c0104650 2024-04-20T21:24:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.432+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8005480 con 0x7f71c0104650 2024-04-20T21:24:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.432+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01a1220 con 0x7f71c0104650 2024-04-20T21:24:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.433+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0069740 con 0x7f71c0104650 2024-04-20T21:24:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.434+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f71b8005620 con 0x7f71c0104650 2024-04-20T21:24:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.434+0000 7f71b6ffd700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 0x7f71ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.435+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f71b80ce030 con 0x7f71c0104650 2024-04-20T21:24:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.435+0000 7f71c5f21700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 0x7f71ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.436+0000 7f71c5f21700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 0x7f71ac0a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71c0103b10 tx=0x7f71bc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.440+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f71b8021020 con 0x7f71c0104650 2024-04-20T21:24:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:24:45.656+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f71c0062e90 con 0x7f71ac0a3c30 2024-04-20T21:24:45.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:45 smithi016 ceph-mon[21409]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:45.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:45 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:45.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:45 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:45.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:45 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:45 smithi007 ceph-mon[16867]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:46.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:45 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:46.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:45 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:46.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:45 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:45 smithi186 ceph-mon[22157]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:46.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:45 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:24:46.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:45 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:24:46.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:45 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi007 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 17094M 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi007 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 17094M 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:24:47.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi007 2024-04-20T21:24:47.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 17094M 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:24:47.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:24:48.184 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:47 smithi007 ceph-mon[16867]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:48.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:47 smithi016 ceph-mon[21409]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:48.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:47 smithi186 ceph-mon[22157]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:48.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:48 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/2883013264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:48.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:48 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:48.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:48 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]': finished 2024-04-20T21:24:48.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:48 smithi016 ceph-mon[21409]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:24:48.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:48 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:24:49.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:48 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/2883013264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:48 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:48 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]': finished 2024-04-20T21:24:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:48 smithi186 ceph-mon[22157]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:24:49.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:48 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:24:49.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:48 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/2883013264' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:49.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:48 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]: dispatch 2024-04-20T21:24:49.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:48 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbdf4c8d-79a8-4077-9214-86fc2a0d993d"}]': finished 2024-04-20T21:24:49.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:48 smithi007 ceph-mon[16867]: osdmap e10: 2 total, 1 up, 2 in 2024-04-20T21:24:49.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:48 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:24:50.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:49 smithi016 ceph-mon[21409]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:50.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:49 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/1843968162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:50.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:49 smithi186 ceph-mon[22157]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:50.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:49 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/1843968162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:49 smithi007 ceph-mon[16867]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:49 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/1843968162' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:24:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:51 smithi016 ceph-mon[21409]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:52.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:51 smithi186 ceph-mon[22157]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:52.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:51 smithi007 ceph-mon[16867]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:54.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:53 smithi016 ceph-mon[21409]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:54.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:53 smithi186 ceph-mon[22157]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:54.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:53 smithi007 ceph-mon[16867]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:56.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:55 smithi016 ceph-mon[21409]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:24:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:55 smithi186 ceph-mon[22157]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:24:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:56.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:55 smithi007 ceph-mon[16867]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-20T21:24:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:24:57.101 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:56 smithi016 ceph-mon[21409]: Deploying daemon osd.1 on smithi016 2024-04-20T21:24:57.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:56 smithi186 ceph-mon[22157]: Deploying daemon osd.1 on smithi016 2024-04-20T21:24:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:56 smithi007 ceph-mon[16867]: Deploying daemon osd.1 on smithi016 2024-04-20T21:24:58.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:57 smithi016 ceph-mon[21409]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:58.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:57 smithi186 ceph-mon[22157]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:24:58.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:57 smithi007 ceph-mon[16867]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:00.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:24:59 smithi016 ceph-mon[21409]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:24:59 smithi186 ceph-mon[22157]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:00.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:24:59 smithi007 ceph-mon[16867]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:02 smithi016 ceph-mon[21409]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:02.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:02 smithi186 ceph-mon[22157]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:02.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:02 smithi007 ceph-mon[16867]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:02.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:02.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:04.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.911+0000 7f71b6ffd700 1 -- 172.21.15.16:0/4143654944 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f71c0062e90 con 0x7f71ac0a3c30 2024-04-20T21:25:04.910 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 1 on host 'smithi016' 2024-04-20T21:25:04.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 msgr2=0x7f71ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 0x7f71ac0a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f71c0103b10 tx=0x7f71bc009190 comp rx=0 tx=0).stop 2024-04-20T21:25:04.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 msgr2=0x7f71c01a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71b800ada0 tx=0x7f71b800c330 comp rx=0 tx=0).stop 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 shutdown_connections 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f71ac0a3c30 0x7f71ac0a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f71c0102ab0 0x7f71c0198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c0104650 0x7f71c01a0320 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 --2- 172.21.15.16:0/4143654944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0103cb0 0x7f71c01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 >> 172.21.15.16:0/4143654944 conn(0x7f71c00fe050 msgr2=0x7f71c0107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 shutdown_connections 2024-04-20T21:25:04.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:25:04.913+0000 7f71c8185700 1 -- 172.21.15.16:0/4143654944 wait complete. 2024-04-20T21:25:05.633 DEBUG:teuthology.orchestra.run.smithi016:osd.1> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.1.service 2024-04-20T21:25:05.635 INFO:tasks.cephadm:Deploying osd.2 on smithi186 with /dev/vg_nvme/lv_4... 2024-04-20T21:25:05.635 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-20T21:25:05.881 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:25:05.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:05 smithi186 ceph-mon[22157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:06.151 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:05 smithi016 ceph-mon[21409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:05 smithi007 ceph-mon[16867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-20T21:25:07.158 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:06 smithi186 ceph-mon[22157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:07.158 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:06 smithi186 ceph-mon[22157]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.159 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:06 smithi186 ceph-mon[22157]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:07.159 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:06 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:07.159 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:06 smithi186 ceph-mon[22157]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-mon[21409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-mon[21409]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-mon[21409]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:07.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-mon[21409]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.195 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1[26005]: 2024-04-20T21:25:06.880+0000 7fdfaf310700 -1 osd.1 0 waiting for initial osdmap 2024-04-20T21:25:07.196 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:06 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1[26005]: 2024-04-20T21:25:06.885+0000 7fdfab103700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:25:07.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:06 smithi007 ceph-mon[16867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-20T21:25:07.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:06 smithi007 ceph-mon[16867]: from='osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:06 smithi007 ceph-mon[16867]: osdmap e11: 2 total, 1 up, 2 in 2024-04-20T21:25:07.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:06 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:07.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:06 smithi007 ceph-mon[16867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:25:07.819 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-20T21:25:07.824 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-20T21:25:07.824 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-20T21:25:07.825 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-20T21:25:07.825 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115221 s, 910 MB/s 2024-04-20T21:25:07.825 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-20T21:25:08.131 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:07 smithi186 ceph-mon[22157]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:08.132 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:07 smithi186 ceph-mon[22157]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:25:08.132 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:07 smithi186 ceph-mon[22157]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:08.132 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:08.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:07 smithi016 ceph-mon[21409]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:08.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:07 smithi016 ceph-mon[21409]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:25:08.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:07 smithi016 ceph-mon[21409]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:08.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:07 smithi007 ceph-mon[16867]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:07 smithi007 ceph-mon[16867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:25:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:07 smithi007 ceph-mon[16867]: osdmap e12: 2 total, 1 up, 2 in 2024-04-20T21:25:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:08.427 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2024-04-20T21:25:08.669 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] boot 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:08.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:08.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:08.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:08.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:08.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:08 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] boot 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:09.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:09.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:09.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:08 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: osd.1 [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] boot 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: osdmap e13: 2 total, 2 up, 2 in 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:25:09.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:09.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:08 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:10.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:09 smithi186 ceph-mon[22157]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:10.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:09 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi016 2024-04-20T21:25:10.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:09 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 17093M 2024-04-20T21:25:10.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:09 smithi186 ceph-mon[22157]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:10.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 -- 172.21.15.186:0/2202749008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 msgr2=0x7f39a0104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/2202749008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a0104140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3994009990 tx=0x7f3994009ca0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 -- 172.21.15.186:0/2202749008 shutdown_connections 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/2202749008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0104680 0x7f39a010af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/2202749008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0102ae0 0x7f39a0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/2202749008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a0104140 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 -- 172.21.15.186:0/2202749008 >> 172.21.15.186:0/2202749008 conn(0x7f39a00fe0a0 msgr2=0x7f39a01004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 -- 172.21.15.186:0/2202749008 shutdown_connections 2024-04-20T21:25:10.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.137+0000 7f39a866a700 1 -- 172.21.15.186:0/2202749008 wait complete. 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 Processor -- start 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 -- start start 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 0x7f39a0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0104680 0x7f39a01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.139 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e7c0 con 0x7f39a0104680 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e640 con 0x7f39a0102ae0 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a866a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010e940 con 0x7f39a0103ce0 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:54216/0 (socket says 172.21.15.186:54216) 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 -- 172.21.15.186:0/748562566 learned_addr learned my addr 172.21.15.186:0/748562566 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a6406700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 0x7f39a0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 -- 172.21.15.186:0/748562566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 msgr2=0x7f39a0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 0x7f39a0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 -- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0104680 msgr2=0x7f39a01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0104680 0x7f39a01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a5c05700 1 -- 172.21.15.186:0/748562566 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a01a0a70 con 0x7f39a0103ce0 2024-04-20T21:25:10.140 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.138+0000 7f39a6406700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 0x7f39a0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f39a5c05700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994027070 con 0x7f39a0103ce0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3994004150 con 0x7f39a0103ce0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399400f6c0 con 0x7f39a0103ce0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a01a0d00 con 0x7f39a0103ce0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.139+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a11d0 con 0x7f39a0103ce0 2024-04-20T21:25:10.141 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.140+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a002df90 con 0x7f39a0103ce0 2024-04-20T21:25:10.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.141+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f399402a070 con 0x7f39a0103ce0 2024-04-20T21:25:10.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.141+0000 7f399f7fe700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 0x7f39880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.142+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f39940d03a0 con 0x7f39a0103ce0 2024-04-20T21:25:10.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.142+0000 7f39a6406700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 0x7f39880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.142 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.143+0000 7f39a6406700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 0x7f39880a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f39a0103b40 tx=0x7f3998009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.144 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.145+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3994031020 con 0x7f39a0103ce0 2024-04-20T21:25:10.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:09 smithi016 ceph-mon[21409]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:10.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:09 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi016 2024-04-20T21:25:10.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:09 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 17093M 2024-04-20T21:25:10.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:09 smithi016 ceph-mon[21409]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:10.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:09 smithi007 ceph-mon[16867]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-20T21:25:10.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:09 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi016 2024-04-20T21:25:10.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:09 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 17093M 2024-04-20T21:25:10.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:09 smithi007 ceph-mon[16867]: osdmap e14: 2 total, 2 up, 2 in 2024-04-20T21:25:10.348 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:10.348+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f39a0062e90 con 0x7f39880a3bb0 2024-04-20T21:25:11.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:11.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:11.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:11.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:11.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:11.203 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:11.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:12.165 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:11 smithi186 ceph-mon[22157]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:12.165 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:11 smithi186 ceph-mon[22157]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:12.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:11 smithi016 ceph-mon[21409]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:12.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:11 smithi016 ceph-mon[21409]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:12.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:11 smithi007 ceph-mon[16867]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:12.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:11 smithi007 ceph-mon[16867]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/3706254418' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]': finished 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:13 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/1854120029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:14.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:14.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/3706254418' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]': finished 2024-04-20T21:25:14.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:14.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:14.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:13 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/1854120029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/3706254418' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]: dispatch 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40aadd77-3cf6-4a09-b30a-b038345607eb"}]': finished 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: osdmap e15: 3 total, 2 up, 3 in 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:13 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/1854120029' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:16.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:15 smithi016 ceph-mon[21409]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:16.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:15 smithi186 ceph-mon[22157]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:15 smithi007 ceph-mon[16867]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:18.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:17 smithi016 ceph-mon[21409]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:18.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:17 smithi186 ceph-mon[22157]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:18.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:17 smithi007 ceph-mon[16867]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:19 smithi007 ceph-mon[16867]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:20.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:19 smithi016 ceph-mon[21409]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:20.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:19 smithi186 ceph-mon[22157]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:21.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:25:21.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:21.056 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:20 smithi186 ceph-mon[22157]: Deploying daemon osd.2 on smithi186 2024-04-20T21:25:21.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:25:21.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:21.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:20 smithi016 ceph-mon[21409]: Deploying daemon osd.2 on smithi186 2024-04-20T21:25:21.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-20T21:25:21.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:21.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:20 smithi007 ceph-mon[16867]: Deploying daemon osd.2 on smithi186 2024-04-20T21:25:22.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:21 smithi186 ceph-mon[22157]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:22.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:21 smithi007 ceph-mon[16867]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:22.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:21 smithi016 ceph-mon[21409]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:24.200 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:23 smithi186 ceph-mon[22157]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:24.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:23 smithi007 ceph-mon[16867]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:24.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:23 smithi016 ceph-mon[21409]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:26.062 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:25 smithi186 ceph-mon[22157]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:26.062 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:25 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.062 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:25 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.062 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:25 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:25 smithi007 ceph-mon[16867]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:25 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:25 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:25 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:26.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:25 smithi016 ceph-mon[21409]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:26.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:25 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:25 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:26.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:25 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:28.124 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:27 smithi186 ceph-mon[22157]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:28.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:27 smithi007 ceph-mon[16867]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:28.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:27 smithi016 ceph-mon[21409]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:29.134 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 2 on host 'smithi186' 2024-04-20T21:25:29.134 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.135+0000 7f399f7fe700 1 -- 172.21.15.186:0/748562566 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39a0062e90 con 0x7f39880a3bb0 2024-04-20T21:25:29.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 msgr2=0x7f39880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:29.136 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 0x7f39880a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f39a0103b40 tx=0x7f3998009570 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 msgr2=0x7f39a01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3994000c00 tx=0x7f3994009ee0 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 shutdown_connections 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a0104680 0x7f39a01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39a0102ae0 0x7f39a0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f39880a3bb0 0x7f39880a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 --2- 172.21.15.186:0/748562566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f39a0103ce0 0x7f39a01993a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 >> 172.21.15.186:0/748562566 conn(0x7f39a00fe0a0 msgr2=0x7f39a0106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 shutdown_connections 2024-04-20T21:25:29.137 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:25:29.137+0000 7f39a866a700 1 -- 172.21.15.186:0/748562566 wait complete. 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:29.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:25:29.757 DEBUG:teuthology.orchestra.run.smithi186:osd.2> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.2.service 2024-04-20T21:25:29.759 INFO:tasks.cephadm:Deploying osd.3 on smithi007 with /dev/vg_nvme/lv_3... 2024-04-20T21:25:29.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:25:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:25:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:30 smithi186 ceph-mon[22157]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:30 smithi186 ceph-mon[22157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:30.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:30 smithi007 ceph-mon[16867]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:30.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:30 smithi007 ceph-mon[16867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:30 smithi016 ceph-mon[21409]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:30 smithi016 ceph-mon[21409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-20T21:25:31.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:31.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:25:31.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:25:31.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:31.396 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.397 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:31 smithi007 ceph-mon[16867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.683 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:31.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:25:31.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:25:31.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:31.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.684 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-mon[22157]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:31.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-20T21:25:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: osdmap e16: 3 total, 2 up, 3 in 2024-04-20T21:25:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: from='osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:31 smithi016 ceph-mon[21409]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:25:31.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2[27296]: 2024-04-20T21:25:31.683+0000 7f961dde3700 -1 osd.2 0 waiting for initial osdmap 2024-04-20T21:25:31.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2[27296]: 2024-04-20T21:25:31.688+0000 7f9615bce700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:25:32.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:32 smithi186 ceph-mon[22157]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:25:32.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:32 smithi186 ceph-mon[22157]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:25:32.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.461 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:32 smithi186 ceph-mon[22157]: from='osd.2 ' entity='osd.2' 2024-04-20T21:25:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:32 smithi007 ceph-mon[16867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:25:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:32 smithi007 ceph-mon[16867]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:25:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:32 smithi007 ceph-mon[16867]: from='osd.2 ' entity='osd.2' 2024-04-20T21:25:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:32 smithi016 ceph-mon[21409]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:25:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:32 smithi016 ceph-mon[21409]: osdmap e17: 3 total, 2 up, 3 in 2024-04-20T21:25:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:32 smithi016 ceph-mon[21409]: from='osd.2 ' entity='osd.2' 2024-04-20T21:25:33.043 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:25:33.043 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:25:33.043 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-04-20T21:25:33.044 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-04-20T21:25:33.044 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329276 s, 318 MB/s 2024-04-20T21:25:33.044 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-04-20T21:25:33.437 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] boot 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi186 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.438 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:33.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:33.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] boot 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi186 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:33.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: osd.2 [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] boot 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: osdmap e18: 3 total, 3 up, 3 in 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi186 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:25:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 17088M 2024-04-20T21:25:33.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:33.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:25:33.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:33.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_3 2024-04-20T21:25:34.322 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:25:35.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:35 smithi016 ceph-mon[21409]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:35.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:35 smithi016 ceph-mon[21409]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:25:35.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:35 smithi186 ceph-mon[22157]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:35 smithi186 ceph-mon[22157]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:25:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:35.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:35 smithi007 ceph-mon[16867]: pgmap v91: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:35.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:35 smithi007 ceph-mon[16867]: osdmap e19: 3 total, 3 up, 3 in 2024-04-20T21:25:35.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:36.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:36.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 ceph-mon[16867]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:25:36.466 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:36.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:36.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 ceph-mon[21409]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:25:36.626 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 -- 172.21.15.7:0/1308347815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc076a20 msgr2=0x7f34bc074e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 --2- 172.21.15.7:0/1308347815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc076a20 0x7f34bc074e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009990 tx=0x7f34b0009ca0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 -- 172.21.15.7:0/1308347815 shutdown_connections 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 --2- 172.21.15.7:0/1308347815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc10a8c0 0x7f34bc10cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 --2- 172.21.15.7:0/1308347815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc076a20 0x7f34bc074e80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 --2- 172.21.15.7:0/1308347815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc0758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.645+0000 7f34c2400700 1 -- 172.21.15.7:0/1308347815 >> 172.21.15.7:0/1308347815 conn(0x7f34bc0fdf80 msgr2=0x7f34bc1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.646+0000 7f34c2400700 1 -- 172.21.15.7:0/1308347815 shutdown_connections 2024-04-20T21:25:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.646+0000 7f34c2400700 1 -- 172.21.15.7:0/1308347815 wait complete. 2024-04-20T21:25:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.647+0000 7f34c2400700 1 Processor -- start 2024-04-20T21:25:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.647+0000 7f34c2400700 1 -- start start 2024-04-20T21:25:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 0x7f34bc19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 0x7f34bc1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1107f0 con 0x7f34bc075450 2024-04-20T21:25:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc110670 con 0x7f34bc10a8c0 2024-04-20T21:25:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c2400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc110970 con 0x7f34bc076a20 2024-04-20T21:25:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 0x7f34bc19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c1000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45814/0 (socket says 172.21.15.7:45814) 2024-04-20T21:25:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c1a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 0x7f34bc1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c1000700 1 -- 172.21.15.7:0/4213712742 learned_addr learned my addr 172.21.15.7:0/4213712742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 -- 172.21.15.7:0/4213712742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 msgr2=0x7f34bc1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 0x7f34bc1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 -- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 msgr2=0x7f34bc19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34bbe00700 1 -- 172.21.15.7:0/4213712742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1a2a00 con 0x7f34bc076a20 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.648+0000 7f34c1a00700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 0x7f34bc1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34bbe00700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 0x7f34bc19b2f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e9f0 tx=0x7f34a800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800c9c0 con 0x7f34bc076a20 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a800cb60 con 0x7f34bc076a20 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1a2cf0 con 0x7f34bc076a20 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8018680 con 0x7f34bc076a20 2024-04-20T21:25:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.649+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a31c0 con 0x7f34bc076a20 2024-04-20T21:25:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.650+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc069740 con 0x7f34bc076a20 2024-04-20T21:25:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.653+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) ==== 138589+0+0 (secure 0 0 0) 0x7f34a8010430 con 0x7f34bc076a20 2024-04-20T21:25:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.654+0000 7f34b9600700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 0x7f34a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.654+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f34a80cefb0 con 0x7f34bc076a20 2024-04-20T21:25:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.654+0000 7f34c1000700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 0x7f34a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.655+0000 7f34c1000700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 0x7f34a40a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b710 tx=0x7f34b000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.656+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f34a809ce30 con 0x7f34bc076a20 2024-04-20T21:25:36.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:36.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 ceph-mon[22157]: osdmap e20: 3 total, 3 up, 3 in 2024-04-20T21:25:36.692 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33755]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33755]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33755]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33755]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33758]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:36.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33758]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.784 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33758]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.784 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33758]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.908 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30000]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:36.908 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30000]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.908 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30000]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.909 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30000]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.909 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30003]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:36.909 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30003]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.909 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30003]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.909 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30003]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:36.924+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f34bc062e90 con 0x7f34a40a3c30 2024-04-20T21:25:36.965 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31345]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31345]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31345]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31345]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31348]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31348]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31348]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.966 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31348]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31351]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:36.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31351]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31351]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31351]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:36.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31354]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:36.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31354]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:36.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31354]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:36.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:36 smithi186 sudo[31354]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30006]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30006]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30006]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30006]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30009]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:37.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:36 smithi016 sudo[30009]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33781]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33781]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33784]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33784]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33784]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-20T21:25:37.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:36 smithi007 sudo[33784]: pam_unix(sudo:session): session closed for user root 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:37.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:37.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:37.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:37.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: osdmap e21: 3 total, 3 up, 3 in 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:25:37.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:25:37.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:25:37.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-20T21:25:37.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:38.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:25:38.348+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f34a8098da0 con 0x7f34bc076a20 2024-04-20T21:25:38.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:38 smithi007 ceph-mon[16867]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:25:38.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:38 smithi016 ceph-mon[21409]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:25:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:38 smithi186 ceph-mon[22157]: osdmap e22: 3 total, 3 up, 3 in 2024-04-20T21:25:39.488 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:39 smithi007 ceph-mon[16867]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:39.488 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:39 smithi007 ceph-mon[16867]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:25:39.488 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:39 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:25:39.488 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:39 smithi007 ceph-mon[16867]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-20T21:25:39.596 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:39 smithi016 ceph-mon[21409]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:39.596 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:39 smithi016 ceph-mon[21409]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:25:39.596 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:39 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:25:39.596 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:39 smithi016 ceph-mon[21409]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-20T21:25:39.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:39 smithi186 ceph-mon[22157]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:39.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:39 smithi186 ceph-mon[22157]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:25:39.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:39 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:25:39.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:39 smithi186 ceph-mon[22157]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/50006700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]': finished 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:25:41.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:41 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:25:41.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:41.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/50006700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]': finished 2024-04-20T21:25:41.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:25:41.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:41 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/50006700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]: dispatch 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3058d99a-73d2-459c-ac66-127d373f59fc"}]': finished 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: osdmap e23: 4 total, 3 up, 4 in 2024-04-20T21:25:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:41 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:25:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:42 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3495328723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:42 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3495328723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:42 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3495328723' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:25:43.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:43 smithi016 ceph-mon[21409]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:43.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:43 smithi186 ceph-mon[22157]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:43.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:43 smithi007 ceph-mon[16867]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:45 smithi016 ceph-mon[21409]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:45 smithi186 ceph-mon[22157]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:45 smithi007 ceph-mon[16867]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:47.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:47 smithi007 ceph-mon[16867]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:47.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:47 smithi016 ceph-mon[21409]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:47 smithi186 ceph-mon[22157]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:49 smithi007 ceph-mon[16867]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:49 smithi016 ceph-mon[21409]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:49.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:49 smithi186 ceph-mon[22157]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:25:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:50 smithi016 ceph-mon[21409]: Deploying daemon osd.3 on smithi007 2024-04-20T21:25:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:25:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:50 smithi186 ceph-mon[22157]: Deploying daemon osd.3 on smithi007 2024-04-20T21:25:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-20T21:25:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:25:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:50 smithi007 ceph-mon[16867]: Deploying daemon osd.3 on smithi007 2024-04-20T21:25:51.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:51 smithi007 ceph-mon[16867]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:51.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:51 smithi016 ceph-mon[21409]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:51.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:51 smithi186 ceph-mon[22157]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:53.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:53 smithi016 ceph-mon[21409]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:53.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:53 smithi186 ceph-mon[22157]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:53 smithi007 ceph-mon[16867]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:55.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:55 smithi016 ceph-mon[21409]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:55.704 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:55 smithi007 ceph-mon[16867]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:55.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:55 smithi186 ceph-mon[22157]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:57.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:57 smithi016 ceph-mon[21409]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:57.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:57 smithi186 ceph-mon[22157]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:25:57.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:57 smithi007 ceph-mon[16867]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:00.153 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:59 smithi007 ceph-mon[16867]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:00.154 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.154 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.154 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:25:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:00.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:59 smithi016 ceph-mon[21409]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:00.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:25:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:59 smithi186 ceph-mon[22157]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:25:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:02.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:01 smithi007 ceph-mon[16867]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:02.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:01 smithi016 ceph-mon[21409]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:02.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:01 smithi186 ceph-mon[22157]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:03 smithi007 ceph-mon[16867]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:03 smithi007 ceph-mon[16867]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.667+0000 7f34b9600700 1 -- 172.21.15.7:0/4213712742 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34bc062e90 con 0x7f34a40a3c30 2024-04-20T21:26:03.667 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 3 on host 'smithi007' 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.670+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 msgr2=0x7f34a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.670+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 0x7f34a40a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b710 tx=0x7f34b000b880 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 msgr2=0x7f34bc19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 0x7f34bc19b2f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f34a800e9f0 tx=0x7f34a800ed00 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 shutdown_connections 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34bc10a8c0 0x7f34bc1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34bc075450 0x7f34bc19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f34a40a3c30 0x7f34a40a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 --2- 172.21.15.7:0/4213712742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f34bc076a20 0x7f34bc19b2f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 >> 172.21.15.7:0/4213712742 conn(0x7f34bc0fdf80 msgr2=0x7f34bc106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 shutdown_connections 2024-04-20T21:26:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:26:03.671+0000 7f34c2400700 1 -- 172.21.15.7:0/4213712742 wait complete. 2024-04-20T21:26:03.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:03 smithi016 ceph-mon[21409]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:03.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:03 smithi016 ceph-mon[21409]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:03.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:03 smithi186 ceph-mon[22157]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:03.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:03.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:03 smithi186 ceph-mon[22157]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:04.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:04 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:04.497 DEBUG:teuthology.orchestra.run.smithi007:osd.3> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.3.service 2024-04-20T21:26:04.502 INFO:tasks.cephadm:Deploying osd.4 on smithi016 with /dev/vg_nvme/lv_3... 2024-04-20T21:26:04.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:04 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: osdmap e24: 4 total, 3 up, 4 in 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:04.772 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:04 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] boot 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:05.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:05.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] boot 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:05.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: from='osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: osdmap e25: 4 total, 3 up, 4 in 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: osd.3 [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] boot 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: osdmap e26: 4 total, 4 up, 4 in 2024-04-20T21:26:05.901 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:26:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:06 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:26:06.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:06 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:26:06.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:06 smithi186 ceph-mon[22157]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:06.770 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:06 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:26:06.770 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:06 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:26:06.770 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:06 smithi016 ceph-mon[21409]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:26:06.774 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:26:06.774 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-20T21:26:06.774 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-20T21:26:06.774 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00954582 s, 1.1 GB/s 2024-04-20T21:26:06.775 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 2024-04-20T21:26:06.775 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-20T21:26:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:06 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:26:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:06 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:26:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:06 smithi007 ceph-mon[16867]: osdmap e27: 4 total, 4 up, 4 in 2024-04-20T21:26:07.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-04-20T21:26:07.671 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:26:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:07 smithi016 ceph-mon[21409]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:07 smithi016 ceph-mon[21409]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:07 smithi186 ceph-mon[22157]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:07 smithi186 ceph-mon[22157]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:07.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:07 smithi007 ceph-mon[16867]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:07 smithi007 ceph-mon[16867]: osdmap e28: 4 total, 4 up, 4 in 2024-04-20T21:26:09.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 -- 172.21.15.16:0/554654307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb0103d40 msgr2=0x7f0cb01041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/554654307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb0103d40 0x7f0cb01041a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009990 tx=0x7f0ca4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 -- 172.21.15.16:0/554654307 shutdown_connections 2024-04-20T21:26:09.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/554654307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb01046e0 0x7f0cb010af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/554654307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cb0100d60 0x7f0cb0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/554654307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb0103d40 0x7f0cb01041a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 -- 172.21.15.16:0/554654307 >> 172.21.15.16:0/554654307 conn(0x7f0cb00fc6e0 msgr2=0x7f0cb00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 -- 172.21.15.16:0/554654307 shutdown_connections 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 -- 172.21.15.16:0/554654307 wait complete. 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.318+0000 7f0cb55ed700 1 Processor -- start 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 -- start start 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cb0100d60 0x7f0cb01036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 0x7f0cb0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010e770 con 0x7f0cb0100d60 2024-04-20T21:26:09.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010e5f0 con 0x7f0cb01046e0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cb55ed700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010e8f0 con 0x7f0cb0103d40 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59454/0 (socket says 172.21.15.16:59454) 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 -- 172.21.15.16:0/1223144978 learned_addr learned my addr 172.21.15.16:0/1223144978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cae7fc700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 0x7f0cb0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 -- 172.21.15.16:0/1223144978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 msgr2=0x7f0cb0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 0x7f0cb0101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 -- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cb0100d60 msgr2=0x7f0cb01036a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cb0100d60 0x7f0cb01036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0caf7fe700 1 -- 172.21.15.16:0/1223144978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb0102ae0 con 0x7f0cb01046e0 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.319+0000 7f0cae7fc700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 0x7f0cb0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:09.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0caf7fe700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800b970 tx=0x7f0ca800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8019070 con 0x7f0cb01046e0 2024-04-20T21:26:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb01a2fa0 con 0x7f0cb01046e0 2024-04-20T21:26:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca8014440 con 0x7f0cb01046e0 2024-04-20T21:26:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca80075d0 con 0x7f0cb01046e0 2024-04-20T21:26:09.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.320+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01a3480 con 0x7f0cb01046e0 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.321+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb0069740 con 0x7f0cb01046e0 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.322+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0ca801c070 con 0x7f0cb01046e0 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.323+0000 7f0c97fff700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 0x7f0c980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.323+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3669+0+0 (secure 0 0 0) 0x7f0ca80cead0 con 0x7f0cb01046e0 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.323+0000 7f0caeffd700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 0x7f0c980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:09.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.324+0000 7f0caeffd700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 0x7f0c980a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cb006a530 tx=0x7f0ca0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.327+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0ca809c6d0 con 0x7f0cb01046e0 2024-04-20T21:26:09.436 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:09 smithi007 ceph-mon[16867]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:09.436 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:09 smithi007 ceph-mon[16867]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:09.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:09 smithi016 ceph-mon[21409]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:09.478 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:09 smithi016 ceph-mon[21409]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:09.546+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0cb0062e90 con 0x7f0c980a3c40 2024-04-20T21:26:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:09 smithi186 ceph-mon[22157]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:09 smithi186 ceph-mon[22157]: osdmap e29: 4 total, 4 up, 4 in 2024-04-20T21:26:10.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:10.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi007 2024-04-20T21:26:10.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 8547M 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:10.347 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.348 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.348 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi007 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:10.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 8547M 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: osdmap e30: 4 total, 4 up, 4 in 2024-04-20T21:26:10.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi007 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 8547M 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:10.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:10.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:10.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:10.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:12.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:11 smithi016 ceph-mon[21409]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:26:12.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:11 smithi186 ceph-mon[22157]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:26:12.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:11 smithi007 ceph-mon[16867]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:26:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:12 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/3764177052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:12 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:12 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]': finished 2024-04-20T21:26:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:12 smithi016 ceph-mon[21409]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:12 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:13.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:12 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/3764177052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:12 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:12 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]': finished 2024-04-20T21:26:13.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:12 smithi186 ceph-mon[22157]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:13.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:12 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:12 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/3764177052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:12 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]: dispatch 2024-04-20T21:26:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:12 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeed2b10-a459-46cc-b744-38c3c4fb5eb2"}]': finished 2024-04-20T21:26:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:12 smithi007 ceph-mon[16867]: osdmap e31: 5 total, 4 up, 5 in 2024-04-20T21:26:13.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:12 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:13 smithi186 ceph-mon[22157]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-20T21:26:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:13 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/2181572045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:14.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:13 smithi016 ceph-mon[21409]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-20T21:26:14.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:13 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/2181572045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:13 smithi007 ceph-mon[16867]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-20T21:26:14.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:13 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/2181572045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:16.178 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:15 smithi016 ceph-mon[21409]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:26:16.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:15 smithi186 ceph-mon[22157]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:26:16.266 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:15 smithi007 ceph-mon[16867]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:26:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:17 smithi016 ceph-mon[21409]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:26:18.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:17 smithi186 ceph-mon[22157]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:26:18.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:17 smithi007 ceph-mon[16867]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:26:20.130 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:19 smithi016 ceph-mon[21409]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:20.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:19 smithi186 ceph-mon[22157]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:20.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:19 smithi007 ceph-mon[16867]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:21.116 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:26:21.116 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:21.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:26:21.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:21.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-20T21:26:21.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:22.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:21 smithi186 ceph-mon[22157]: Deploying daemon osd.4 on smithi016 2024-04-20T21:26:22.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:21 smithi186 ceph-mon[22157]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:22.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:21 smithi016 ceph-mon[21409]: Deploying daemon osd.4 on smithi016 2024-04-20T21:26:22.255 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:21 smithi016 ceph-mon[21409]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:22.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:21 smithi007 ceph-mon[16867]: Deploying daemon osd.4 on smithi016 2024-04-20T21:26:22.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:21 smithi007 ceph-mon[16867]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:24.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:23 smithi016 ceph-mon[21409]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:24.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:23 smithi186 ceph-mon[22157]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:24.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:23 smithi007 ceph-mon[16867]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:26.134 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:25 smithi016 ceph-mon[21409]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:25 smithi186 ceph-mon[22157]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:25 smithi007 ceph-mon[16867]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:27 smithi007 ceph-mon[16867]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:27 smithi016 ceph-mon[21409]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:27 smithi186 ceph-mon[22157]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:29.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:29 smithi007 ceph-mon[16867]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:29.548 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:29 smithi016 ceph-mon[21409]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:29.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:29 smithi186 ceph-mon[22157]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:30.241 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 4 on host 'smithi016' 2024-04-20T21:26:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.241+0000 7f0c97fff700 1 -- 172.21.15.16:0/1223144978 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0cb0062e90 con 0x7f0c980a3c40 2024-04-20T21:26:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 msgr2=0x7f0c980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 0x7f0c980a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0cb006a530 tx=0x7f0ca0006d20 comp rx=0 tx=0).stop 2024-04-20T21:26:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 msgr2=0x7f0cb0102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:30.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800b970 tx=0x7f0ca800bd30 comp rx=0 tx=0).stop 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 shutdown_connections 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cb01046e0 0x7f0cb0102230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cb0100d60 0x7f0cb01036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0c980a3c40 0x7f0c980a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 --2- 172.21.15.16:0/1223144978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0cb0103d40 0x7f0cb0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 >> 172.21.15.16:0/1223144978 conn(0x7f0cb00fc6e0 msgr2=0x7f0cb01070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 shutdown_connections 2024-04-20T21:26:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:26:30.244+0000 7f0cb55ed700 1 -- 172.21.15.16:0/1223144978 wait complete. 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:30.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:30.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:30.857 DEBUG:teuthology.orchestra.run.smithi016:osd.4> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.4.service 2024-04-20T21:26:30.859 INFO:tasks.cephadm:Deploying osd.5 on smithi186 with /dev/vg_nvme/lv_3... 2024-04-20T21:26:30.860 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-20T21:26:31.106 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:26:31.359 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4[32813]: 2024-04-20T21:26:31.356+0000 7ff3e851c700 -1 osd.4 0 waiting for initial osdmap 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:31.554 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:31 smithi186 ceph-mon[22157]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:31.694 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4[32813]: 2024-04-20T21:26:31.363+0000 7ff3e430f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:31 smithi016 ceph-mon[21409]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: from='osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: osdmap e32: 5 total, 4 up, 5 in 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:31 smithi007 ceph-mon[16867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:26:32.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:32 smithi186 ceph-mon[22157]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:26:32.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:32 smithi186 ceph-mon[22157]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:26:32.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:32.456 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:32 smithi016 ceph-mon[21409]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:26:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:32 smithi016 ceph-mon[21409]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:26:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:32 smithi007 ceph-mon[16867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:26:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:32 smithi007 ceph-mon[16867]: osdmap e33: 5 total, 4 up, 5 in 2024-04-20T21:26:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113687 s, 922 MB/s 2024-04-20T21:26:33.036 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-20T21:26:33.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:26:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:26:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] boot 2024-04-20T21:26:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:26:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] boot 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:26:33.657 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:33.685 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: osd.4 [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] boot 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: osdmap e34: 5 total, 5 up, 5 in 2024-04-20T21:26:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:26:33.933 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi016 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 8546M 2024-04-20T21:26:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi016 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 8546M 2024-04-20T21:26:34.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:34.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:34.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: osdmap e35: 5 total, 5 up, 5 in 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi016 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 8546M 2024-04-20T21:26:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:26:34.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:35.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.489+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2378082848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 msgr2=0x7f1cf4102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.489+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2378082848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4102110 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00099c0 tx=0x7f1ce0009cd0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2378082848 shutdown_connections 2024-04-20T21:26:35.491 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2378082848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102650 0x7f1cf4109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2378082848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cf4102f70 0x7f1cf4103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2378082848 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4102110 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2378082848 >> 172.21.15.186:0/2378082848 conn(0x7f1cf40fc700 msgr2=0x7f1cf40feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2378082848 shutdown_connections 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2378082848 wait complete. 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 Processor -- start 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- start start 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cf4102650 0x7f1cf4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102f70 0x7f1cf41a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d660 con 0x7f1cf4102650 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d4e0 con 0x7f1cf4102f70 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cfadd0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf410d7e0 con 0x7f1cf4101cb0 2024-04-20T21:26:35.492 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cf8b6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cf8b6c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:58216/0 (socket says 172.21.15.186:58216) 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.490+0000 7f1cf8b6c700 1 -- 172.21.15.186:0/2991275871 learned_addr learned my addr 172.21.15.186:0/2991275871 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf936d700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102f70 0x7f1cf41a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 -- 172.21.15.186:0/2991275871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102f70 msgr2=0x7f1cf41a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102f70 0x7f1cf41a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 -- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cf4102650 msgr2=0x7f1cf4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cf4102650 0x7f1cf4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 -- 172.21.15.186:0/2991275871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf41a0a60 con 0x7f1cf4101cb0 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf8b6c700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b530 tx=0x7f1ce800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.493 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8015020 con 0x7f1cf4101cb0 2024-04-20T21:26:35.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf41a0d50 con 0x7f1cf4101cb0 2024-04-20T21:26:35.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8011d10 con 0x7f1cf4101cb0 2024-04-20T21:26:35.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800f2e0 con 0x7f1cf4101cb0 2024-04-20T21:26:35.494 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.491+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41a1280 con 0x7f1cf4101cb0 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.492+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf4069770 con 0x7f1cf4101cb0 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.493+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f1ce8018020 con 0x7f1cf4101cb0 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.494+0000 7f1cf1ffb700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 0x7f1ce40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.494+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f1ce80ceb70 con 0x7f1cf4101cb0 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.494+0000 7f1cf3fff700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 0x7f1ce40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.495 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.495+0000 7f1cf3fff700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 0x7f1ce40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00098e0 tx=0x7f1ce0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.498 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.499+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1cf4069770 con 0x7f1cf4101cb0 2024-04-20T21:26:35.704 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:35.704+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1cf40fe870 con 0x7f1ce40a3c40 2024-04-20T21:26:35.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:35 smithi186 ceph-mon[22157]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:35.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:35.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:35.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:35 smithi007 ceph-mon[16867]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:35 smithi016 ceph-mon[21409]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:26:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:26:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:37.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:36 smithi186 ceph-mon[22157]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:37.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:36 smithi007 ceph-mon[16867]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:37.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:36 smithi016 ceph-mon[21409]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:26:38.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:37 smithi186 ceph-mon[22157]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:38.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:37 smithi007 ceph-mon[16867]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:37 smithi016 ceph-mon[21409]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:39.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:38 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/3015793001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:38 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:38 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]': finished 2024-04-20T21:26:39.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:38 smithi186 ceph-mon[22157]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:26:39.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:38 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:39.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:38 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/3015793001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:38 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:38 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]': finished 2024-04-20T21:26:39.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:38 smithi007 ceph-mon[16867]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:26:39.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:38 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:39.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:38 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/3015793001' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:38 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]: dispatch 2024-04-20T21:26:39.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:38 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "19d66739-ba9a-49b9-a99c-7792746f048d"}]': finished 2024-04-20T21:26:39.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:38 smithi016 ceph-mon[21409]: osdmap e36: 6 total, 5 up, 6 in 2024-04-20T21:26:39.346 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:38 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:40.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:39 smithi007 ceph-mon[16867]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:40.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/2985849575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:40.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:39 smithi016 ceph-mon[21409]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:40.486 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:39 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/2985849575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:40.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:39 smithi186 ceph-mon[22157]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:40.487 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:39 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/2985849575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:26:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:41 smithi007 ceph-mon[16867]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:42.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:41 smithi016 ceph-mon[21409]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:41 smithi186 ceph-mon[22157]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:44.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:43 smithi007 ceph-mon[16867]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:44.371 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:43 smithi186 ceph-mon[22157]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:44.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:43 smithi016 ceph-mon[21409]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:46.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:45 smithi186 ceph-mon[22157]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:46.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:45 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:26:46.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:45 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:46.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:45 smithi007 ceph-mon[16867]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:46.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:45 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:26:46.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:45 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:46.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:45 smithi016 ceph-mon[21409]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:46.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:45 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-20T21:26:46.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:45 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:26:47.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:47 smithi007 ceph-mon[16867]: Deploying daemon osd.5 on smithi186 2024-04-20T21:26:47.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:47 smithi016 ceph-mon[21409]: Deploying daemon osd.5 on smithi186 2024-04-20T21:26:47.455 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:47 smithi186 ceph-mon[22157]: Deploying daemon osd.5 on smithi186 2024-04-20T21:26:48.154 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:48 smithi186 ceph-mon[22157]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:48 smithi007 ceph-mon[16867]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:48.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:48 smithi016 ceph-mon[21409]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:50.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:50 smithi186 ceph-mon[22157]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:50 smithi007 ceph-mon[16867]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:50 smithi016 ceph-mon[21409]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:52.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:52 smithi186 ceph-mon[22157]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:52.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.272 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.273 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:52.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:52 smithi007 ceph-mon[16867]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:52.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:52.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:52 smithi016 ceph-mon[21409]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:52.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:52.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:26:54.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:54 smithi007 ceph-mon[16867]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:54.406 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:54 smithi186 ceph-mon[22157]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:54.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:54 smithi016 ceph-mon[21409]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:55.191 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 5 on host 'smithi186' 2024-04-20T21:26:55.191 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.191+0000 7f1cf1ffb700 1 -- 172.21.15.186:0/2991275871 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1cf40fe870 con 0x7f1ce40a3c40 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 msgr2=0x7f1ce40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 0x7f1ce40a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00098e0 tx=0x7f1ce0000bc0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 msgr2=0x7f1cf4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b530 tx=0x7f1ce800b8f0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 shutdown_connections 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1ce40a3c40 0x7f1ce40a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cf4101cb0 0x7f1cf4198e00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cf4102f70 0x7f1cf41a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 --2- 172.21.15.186:0/2991275871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cf4102650 0x7f1cf4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:55.193 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.193+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 >> 172.21.15.186:0/2991275871 conn(0x7f1cf40fc700 msgr2=0x7f1cf40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:55.194 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.194+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 shutdown_connections 2024-04-20T21:26:55.194 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:26:55.194+0000 7f1cfadd0700 1 -- 172.21.15.186:0/2991275871 wait complete. 2024-04-20T21:26:55.795 DEBUG:teuthology.orchestra.run.smithi186:osd.5> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.5.service 2024-04-20T21:26:55.798 INFO:tasks.cephadm:Deploying osd.6 on smithi007 with /dev/vg_nvme/lv_2... 2024-04-20T21:26:55.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.295 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:56.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-20T21:26:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:57 smithi016 ceph-mon[21409]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-mon[22157]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:57.466 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5[34231]: 2024-04-20T21:26:57.191+0000 7f6f6cf79700 -1 osd.5 0 waiting for initial osdmap 2024-04-20T21:26:57.467 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:26:57 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5[34231]: 2024-04-20T21:26:57.203+0000 7f6f68d6c700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: osdmap e37: 6 total, 5 up, 6 in 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: from='osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:26:57.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:57 smithi007 ceph-mon[16867]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-20T21:26:58.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:26:58.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:26:58.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.439 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] boot 2024-04-20T21:26:58.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T21:26:58.440 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] boot 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T21:26:58.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: osdmap e38: 6 total, 5 up, 6 in 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: osd.5 [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] boot 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: osdmap e39: 6 total, 6 up, 6 in 2024-04-20T21:26:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0346903 s, 302 MB/s 2024-04-20T21:26:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-04-20T21:26:59.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:59 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:26:59.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:59 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:26:59.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:59 smithi016 ceph-mon[21409]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:26:59.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:26:59 smithi016 ceph-mon[21409]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:26:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:59 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:26:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:59 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:26:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:59 smithi186 ceph-mon[22157]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:26:59.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:26:59 smithi186 ceph-mon[22157]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:26:59.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:59 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:26:59.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:59 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:26:59.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:59 smithi007 ceph-mon[16867]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:26:59.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:26:59 smithi007 ceph-mon[16867]: osdmap e40: 6 total, 6 up, 6 in 2024-04-20T21:27:00.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_2 2024-04-20T21:27:00.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi186 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi186 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: osdmap e41: 6 total, 6 up, 6 in 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi186 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 8544M 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:27:01.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:01 smithi007 ceph-mon[16867]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:01.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:01 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:01.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:01 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:01.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:01 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:01.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:01 smithi007 ceph-mon[16867]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:01.674 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:01 smithi016 ceph-mon[21409]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:01.674 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:01 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:01.674 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:01 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:01.674 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:01 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:01.674 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:01 smithi016 ceph-mon[21409]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:01.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:01 smithi186 ceph-mon[22157]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:01 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:01 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:01 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:01 smithi186 ceph-mon[22157]: osdmap e42: 6 total, 6 up, 6 in 2024-04-20T21:27:02.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:02 smithi186 ceph-mon[22157]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:02 smithi007 ceph-mon[16867]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:02.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:02 smithi016 ceph-mon[21409]: osdmap e43: 6 total, 6 up, 6 in 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 -- 172.21.15.7:0/2477998747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 msgr2=0x7f58fc10a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 --2- 172.21.15.7:0/2477998747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc10a760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009990 tx=0x7f58e8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 -- 172.21.15.7:0/2477998747 shutdown_connections 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 --2- 172.21.15.7:0/2477998747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 0x7f58fc111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 --2- 172.21.15.7:0/2477998747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 --2- 172.21.15.7:0/2477998747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc10a760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 -- 172.21.15.7:0/2477998747 >> 172.21.15.7:0/2477998747 conn(0x7f58fc0780d0 msgr2=0x7f58fc07a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:02.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 -- 172.21.15.7:0/2477998747 shutdown_connections 2024-04-20T21:27:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.000+0000 7f5903a00700 1 -- 172.21.15.7:0/2477998747 wait complete. 2024-04-20T21:27:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.001+0000 7f5903a00700 1 Processor -- start 2024-04-20T21:27:03.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 -- start start 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 0x7f58fc1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc114d40 con 0x7f58fc10aca0 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc114bc0 con 0x7f58fc109100 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5903a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc114ec0 con 0x7f58fc10a300 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f58fbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:03.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f58fbe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40968/0 (socket says 172.21.15.7:40968) 2024-04-20T21:27:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f58fbe00700 1 -- 172.21.15.7:0/2249165106 learned_addr learned my addr 172.21.15.7:0/2249165106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:27:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5901000700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:03.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.002+0000 7f5901a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 0x7f58fc1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 -- 172.21.15.7:0/2249165106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 msgr2=0x7f58fc19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 -- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 msgr2=0x7f58fc1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 0x7f58fc1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 -- 172.21.15.7:0/2249165106 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc1a6fc0 con 0x7f58fc10a300 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f5901000700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58fbe00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b820 tx=0x7f58e800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8027070 con 0x7f58fc10a300 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58e800fe20 con 0x7f58fc10a300 2024-04-20T21:27:03.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc1a7250 con 0x7f58fc10a300 2024-04-20T21:27:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.003+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017650 con 0x7f58fc10a300 2024-04-20T21:27:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.004+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc1a7700 con 0x7f58fc10a300 2024-04-20T21:27:03.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.005+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc069740 con 0x7f58fc10a300 2024-04-20T21:27:03.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.007+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f58e802a070 con 0x7f58fc10a300 2024-04-20T21:27:03.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.008+0000 7f58f9600700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:03.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.009+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f58e80d0910 con 0x7f58fc10a300 2024-04-20T21:27:03.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.009+0000 7f5901000700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:03.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.009+0000 7f5901000700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 0x7f58e40a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58fc10a160 tx=0x7f58f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:03.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.010+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f58e8031070 con 0x7f58fc10a300 2024-04-20T21:27:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:03.251+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f58fc062e90 con 0x7f58e40a3cc0 2024-04-20T21:27:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:03 smithi007 ceph-mon[16867]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:03.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:03 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:03.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:03 smithi186 ceph-mon[22157]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:03.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:03.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:03.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:03 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:03.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:03 smithi016 ceph-mon[21409]: pgmap v160: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:03.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:03.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:03.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:03 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:04 smithi007 ceph-mon[16867]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:04 smithi186 ceph-mon[22157]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:04 smithi016 ceph-mon[21409]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:05.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:05 smithi007 ceph-mon[16867]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T21:27:05.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:05 smithi016 ceph-mon[21409]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T21:27:05.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:05 smithi186 ceph-mon[22157]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-20T21:27:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T21:27:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/494904192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]': finished 2024-04-20T21:27:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T21:27:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T21:27:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/494904192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]': finished 2024-04-20T21:27:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T21:27:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-20T21:27:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/494904192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]: dispatch 2024-04-20T21:27:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b410e07-62a3-487c-b3fb-0e71d0ed25a4"}]': finished 2024-04-20T21:27:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: osdmap e44: 7 total, 6 up, 7 in 2024-04-20T21:27:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/97913991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/97913991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:08.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/97913991' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:09.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:09 smithi007 ceph-mon[16867]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-20T21:27:09.846 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:09 smithi016 ceph-mon[21409]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-20T21:27:09.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:09 smithi186 ceph-mon[22157]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-20T21:27:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:11 smithi007 ceph-mon[16867]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:27:11.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:11 smithi016 ceph-mon[21409]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:27:11.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:11 smithi186 ceph-mon[22157]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-20T21:27:13.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:13 smithi186 ceph-mon[22157]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:13.757 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:13 smithi007 ceph-mon[16867]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:13 smithi016 ceph-mon[21409]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-20T21:27:15.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:15 smithi007 ceph-mon[16867]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:15.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:15 smithi016 ceph-mon[21409]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:15.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:15 smithi186 ceph-mon[22157]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:17 smithi007 ceph-mon[16867]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:17 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:17 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:17 smithi007 ceph-mon[16867]: Deploying daemon osd.6 on smithi007 2024-04-20T21:27:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:17 smithi016 ceph-mon[21409]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:17 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:17 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:17 smithi016 ceph-mon[21409]: Deploying daemon osd.6 on smithi007 2024-04-20T21:27:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:17 smithi186 ceph-mon[22157]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:17 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-20T21:27:17.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:17 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:17.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:17 smithi186 ceph-mon[22157]: Deploying daemon osd.6 on smithi007 2024-04-20T21:27:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:19 smithi007 ceph-mon[16867]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:19.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:19 smithi016 ceph-mon[21409]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:19 smithi186 ceph-mon[22157]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:21 smithi007 ceph-mon[16867]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:21 smithi016 ceph-mon[21409]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:21 smithi186 ceph-mon[22157]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:23.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:23 smithi007 ceph-mon[16867]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:23.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:23 smithi016 ceph-mon[21409]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:23 smithi186 ceph-mon[22157]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:25.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:25 smithi007 ceph-mon[16867]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:25.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:25 smithi016 ceph-mon[21409]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:25.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:25 smithi186 ceph-mon[22157]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:26.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:26.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:26.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:27.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:27.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:27.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:27.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:27 smithi007 ceph-mon[16867]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:28.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:27 smithi016 ceph-mon[21409]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:28.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:27 smithi186 ceph-mon[22157]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:29.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:29 smithi007 ceph-mon[16867]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:29 smithi007 ceph-mon[16867]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:29 smithi007 ceph-mon[16867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:30.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:29 smithi016 ceph-mon[21409]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:30.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:30.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:30.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:29 smithi016 ceph-mon[21409]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:30.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:29 smithi016 ceph-mon[21409]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:30.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:29 smithi186 ceph-mon[22157]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:30.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:30.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:30.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:29 smithi186 ceph-mon[22157]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:30.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:29 smithi186 ceph-mon[22157]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-20T21:27:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.856+0000 7f58f9600700 1 -- 172.21.15.7:0/2249165106 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f58fc062e90 con 0x7f58e40a3cc0 2024-04-20T21:27:30.857 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 6 on host 'smithi007' 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 msgr2=0x7f58e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 0x7f58e40a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58fc10a160 tx=0x7f58f0009260 comp rx=0 tx=0).stop 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 msgr2=0x7f58fc19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b820 tx=0x7f58e800f690 comp rx=0 tx=0).stop 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 shutdown_connections 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58fc10aca0 0x7f58fc1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc109100 0x7f58fc19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f58e40a3cc0 0x7f58e40a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 --2- 172.21.15.7:0/2249165106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58fc10a300 0x7f58fc19f8a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 >> 172.21.15.7:0/2249165106 conn(0x7f58fc0780d0 msgr2=0x7f58fc10d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 shutdown_connections 2024-04-20T21:27:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:27:30.860+0000 7f5903a00700 1 -- 172.21.15.7:0/2249165106 wait complete. 2024-04-20T21:27:31.770 DEBUG:teuthology.orchestra.run.smithi007:osd.6> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.6.service 2024-04-20T21:27:31.775 INFO:tasks.cephadm:Deploying osd.7 on smithi016 with /dev/vg_nvme/lv_2... 2024-04-20T21:27:31.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:27:31.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:31.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:31.803 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: osdmap e45: 7 total, 6 up, 7 in 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:31.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:27:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:32 smithi007 ceph-mon[16867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:27:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:32 smithi007 ceph-mon[16867]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:27:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:32.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:32 smithi186 ceph-mon[22157]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:27:32.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:32 smithi186 ceph-mon[22157]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:27:32.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:32.868 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:32 smithi016 ceph-mon[21409]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:27:32.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:32 smithi016 ceph-mon[21409]: osdmap e46: 7 total, 6 up, 7 in 2024-04-20T21:27:32.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:32.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:33.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:27:33.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:27:33.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:33.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] boot 2024-04-20T21:27:33.639 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T21:27:33.640 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:27:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:27:33.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:33.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] boot 2024-04-20T21:27:33.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T21:27:33.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:27:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:27:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-04-20T21:27:33.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: osd.6 [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] boot 2024-04-20T21:27:33.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: osdmap e47: 7 total, 7 up, 7 in 2024-04-20T21:27:33.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:27:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:27:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:27:34.046 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-20T21:27:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-20T21:27:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00927989 s, 1.1 GB/s 2024-04-20T21:27:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-20T21:27:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:34 smithi016 ceph-mon[21409]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:27:34.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-04-20T21:27:34.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:34 smithi186 ceph-mon[22157]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:27:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:27:35.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:34 smithi007 ceph-mon[16867]: osdmap e48: 7 total, 7 up, 7 in 2024-04-20T21:27:35.825 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:35 smithi007 ceph-mon[16867]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:35.842 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:35 smithi016 ceph-mon[21409]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:35.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:35 smithi186 ceph-mon[22157]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- 172.21.15.16:0/1573687263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 msgr2=0x7ff46810af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/1573687263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff46810af20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff46000b0d0 tx=0x7ff46000b3e0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- 172.21.15.16:0/1573687263 shutdown_connections 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/1573687263 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff46810af20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/1573687263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468102ad0 0x7ff468102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/1573687263 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468103cd0 0x7ff468104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- 172.21.15.16:0/1573687263 >> 172.21.15.16:0/1573687263 conn(0x7ff4680fe070 msgr2=0x7ff4681004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- 172.21.15.16:0/1573687263 shutdown_connections 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- 172.21.15.16:0/1573687263 wait complete. 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 Processor -- start 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.664+0000 7ff46f1a6700 1 -- start start 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468102ad0 0x7ff468198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 0x7ff468199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810e750 con 0x7ff468103cd0 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810e5d0 con 0x7ff468104670 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46f1a6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff46810e8d0 con 0x7ff468102ad0 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48896/0 (socket says 172.21.15.16:48896) 2024-04-20T21:27:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 -- 172.21.15.16:0/3468848339 learned_addr learned my addr 172.21.15.16:0/3468848339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46cf42700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468102ad0 0x7ff468198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff467fff700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 0x7ff468199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 -- 172.21.15.16:0/3468848339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468102ad0 msgr2=0x7ff468198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468102ad0 0x7ff468198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 -- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 msgr2=0x7ff468199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 0x7ff468199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 -- 172.21.15.16:0/3468848339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681a0950 con 0x7ff468104670 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff467fff700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 0x7ff468199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.665+0000 7ff46d743700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff460006070 tx=0x7ff460007dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.666+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460021020 con 0x7ff468104670 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.666+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff46001d950 con 0x7ff468104670 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.666+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681a0be0 con 0x7ff468104670 2024-04-20T21:27:36.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.666+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff460005510 con 0x7ff468104670 2024-04-20T21:27:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.666+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681a1170 con 0x7ff468104670 2024-04-20T21:27:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.667+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff468069740 con 0x7ff468104670 2024-04-20T21:27:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.669+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff460024020 con 0x7ff468104670 2024-04-20T21:27:36.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.670+0000 7ff465ffb700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 0x7ff4580a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.670+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7ff46000f020 con 0x7ff468104670 2024-04-20T21:27:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.670+0000 7ff46cf42700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 0x7ff4580a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.671+0000 7ff46cf42700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 0x7ff4580a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006fd0 tx=0x7ff45c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.674+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff46009e870 con 0x7ff468104670 2024-04-20T21:27:36.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:36.893+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff468062e90 con 0x7ff4580a3cc0 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi007 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 5698M 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:37.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:37.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:37 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi007 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 5698M 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:38.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:38.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:37 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:38.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi007 2024-04-20T21:27:38.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 5698M 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:27:38.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:27:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:38.218 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:37 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/2960825300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]': finished 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T21:27:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:40.115 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:40.115 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/2960825300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.115 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.115 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]': finished 2024-04-20T21:27:40.115 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T21:27:40.116 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:40.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:40.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/2960825300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]: dispatch 2024-04-20T21:27:40.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af75979c-c869-4b74-a32f-975d039d77ff"}]': finished 2024-04-20T21:27:40.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: osdmap e49: 8 total, 7 up, 8 in 2024-04-20T21:27:40.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:41.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:40 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/3119538482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:41.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:40 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/3119538482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:41.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:40 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/3119538482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:27:42.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:41 smithi016 ceph-mon[21409]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:42.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:41 smithi186 ceph-mon[22157]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:42.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:41 smithi007 ceph-mon[16867]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:43.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:43 smithi186 ceph-mon[22157]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:44.033 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:43 smithi016 ceph-mon[21409]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:44.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:43 smithi007 ceph-mon[16867]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:46.006 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:45 smithi016 ceph-mon[21409]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:46.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:45 smithi186 ceph-mon[22157]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:46.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:45 smithi007 ceph-mon[16867]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:48.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:47 smithi007 ceph-mon[16867]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:48.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:27:48.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:48.086 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:47 smithi007 ceph-mon[16867]: Deploying daemon osd.7 on smithi016 2024-04-20T21:27:48.152 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:47 smithi016 ceph-mon[21409]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:48.152 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:27:48.152 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:48.152 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:47 smithi016 ceph-mon[21409]: Deploying daemon osd.7 on smithi016 2024-04-20T21:27:48.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:47 smithi186 ceph-mon[22157]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-20T21:27:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:48.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:47 smithi186 ceph-mon[22157]: Deploying daemon osd.7 on smithi016 2024-04-20T21:27:50.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:49 smithi007 ceph-mon[16867]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:50.088 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:49 smithi016 ceph-mon[21409]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:50.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:49 smithi186 ceph-mon[22157]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:51.939 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:51 smithi016 ceph-mon[21409]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:52.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:51 smithi186 ceph-mon[22157]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:52.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:51 smithi007 ceph-mon[16867]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:54.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:53 smithi016 ceph-mon[21409]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:54.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:54.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:54.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:54.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:53 smithi186 ceph-mon[22157]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:54.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:54.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:54.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:54.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:53 smithi007 ceph-mon[16867]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:54.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:54.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:54.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:56.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:27:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:27:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:56.680 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 7 on host 'smithi016' 2024-04-20T21:27:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.681+0000 7ff465ffb700 1 -- 172.21.15.16:0/3468848339 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff468062e90 con 0x7ff4580a3cc0 2024-04-20T21:27:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 msgr2=0x7ff4580a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 0x7ff4580a6180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006fd0 tx=0x7ff45c008040 comp rx=0 tx=0).stop 2024-04-20T21:27:56.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 msgr2=0x7ff4681a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff460006070 tx=0x7ff460007dc0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 shutdown_connections 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff4580a3cc0 0x7ff4580a6180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff468102ad0 0x7ff468198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff468104670 0x7ff4681a0280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 --2- 172.21.15.16:0/3468848339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff468103cd0 0x7ff468199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 >> 172.21.15.16:0/3468848339 conn(0x7ff4680fe070 msgr2=0x7ff468106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.683+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 shutdown_connections 2024-04-20T21:27:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:27:56.684+0000 7ff46f1a6700 1 -- 172.21.15.16:0/3468848339 wait complete. 2024-04-20T21:27:57.341 DEBUG:teuthology.orchestra.run.smithi016:osd.7> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.7.service 2024-04-20T21:27:57.344 INFO:tasks.cephadm:Deploying osd.8 on smithi186 with /dev/vg_nvme/lv_2... 2024-04-20T21:27:57.344 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-20T21:27:57.594 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:57.910 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:57 smithi016 ceph-mon[21409]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:57.964 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:57 smithi186 ceph-mon[22157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:58.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:57 smithi007 ceph-mon[16867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-20T21:27:58.879 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-mon[21409]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:27:58.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-mon[21409]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:58.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-mon[21409]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:27:58.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:58.880 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-mon[21409]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:58.880 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7[39375]: 2024-04-20T21:27:58.657+0000 7fc8efd3f700 -1 osd.7 0 waiting for initial osdmap 2024-04-20T21:27:58.880 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:27:58 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7[39375]: 2024-04-20T21:27:58.667+0000 7fc8ebb32700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:27:58.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:58 smithi186 ceph-mon[22157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:27:58.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:58 smithi186 ceph-mon[22157]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:58.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:58 smithi186 ceph-mon[22157]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:27:58.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:58.939 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:58 smithi186 ceph-mon[22157]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:59.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:58 smithi007 ceph-mon[16867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-20T21:27:59.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:58 smithi007 ceph-mon[16867]: from='osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:59.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:58 smithi007 ceph-mon[16867]: osdmap e50: 8 total, 7 up, 8 in 2024-04-20T21:27:59.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:59.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:58 smithi007 ceph-mon[16867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:27:59.684 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-20T21:27:59.684 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-20T21:27:59.684 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-20T21:27:59.684 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-20T21:27:59.685 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124631 s, 841 MB/s 2024-04-20T21:27:59.685 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-20T21:27:59.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:59 smithi007 ceph-mon[16867]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:59.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:59 smithi007 ceph-mon[16867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:27:59.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:59 smithi007 ceph-mon[16867]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:27:59.921 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:27:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:59 smithi016 ceph-mon[21409]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:59 smithi016 ceph-mon[21409]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:27:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:59 smithi016 ceph-mon[21409]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:27:59.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:27:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:27:59.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:59 smithi186 ceph-mon[22157]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-20T21:27:59.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:59 smithi186 ceph-mon[22157]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:27:59.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:59 smithi186 ceph-mon[22157]: osdmap e51: 8 total, 7 up, 8 in 2024-04-20T21:27:59.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:27:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:00.373 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2024-04-20T21:28:00.616 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] boot 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:00 smithi016 ceph-mon[21409]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] boot 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:00.917 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:00.918 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:00 smithi186 ceph-mon[22157]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:28:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:28:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: osd.7 [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] boot 2024-04-20T21:28:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: osdmap e52: 8 total, 8 up, 8 in 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:01.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:00 smithi007 ceph-mon[16867]: osdmap e53: 8 total, 8 up, 8 in 2024-04-20T21:28:01.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:01 smithi186 ceph-mon[22157]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:01.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:01 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi016 2024-04-20T21:28:01.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:01 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 5697M 2024-04-20T21:28:01.776 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:01 smithi186 ceph-mon[22157]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:01.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:01 smithi016 ceph-mon[21409]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:01 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi016 2024-04-20T21:28:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:01 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 5697M 2024-04-20T21:28:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:01 smithi016 ceph-mon[21409]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:02.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:01 smithi007 ceph-mon[16867]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:02.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:01 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi016 2024-04-20T21:28:02.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:01 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 5697M 2024-04-20T21:28:02.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:01 smithi007 ceph-mon[16867]: osdmap e54: 8 total, 8 up, 8 in 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 -- 172.21.15.186:0/470475052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404103e40 msgr2=0x7f24041042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 --2- 172.21.15.186:0/470475052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404103e40 0x7f24041042a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009990 tx=0x7f23f8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 -- 172.21.15.186:0/470475052 shutdown_connections 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 --2- 172.21.15.186:0/470475052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24041047e0 0x7f240410b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 --2- 172.21.15.186:0/470475052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2404100dc0 0x7f24041011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 --2- 172.21.15.186:0/470475052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404103e40 0x7f24041042a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 -- 172.21.15.186:0/470475052 >> 172.21.15.186:0/470475052 conn(0x7f24040fc720 msgr2=0x7f24040feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 -- 172.21.15.186:0/470475052 shutdown_connections 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 -- 172.21.15.186:0/470475052 wait complete. 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.243+0000 7f2409ec7700 1 Processor -- start 2024-04-20T21:28:02.244 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 -- start start 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2404103e40 0x7f2404199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24041047e0 0x7f24041a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e7d0 con 0x7f24041047e0 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e650 con 0x7f2404103e40 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2409ec7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e950 con 0x7f2404100dc0 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:55346/0 (socket says 172.21.15.186:55346) 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 -- 172.21.15.186:0/919801237 learned_addr learned my addr 172.21.15.186:0/919801237 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f2402ffd700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2404103e40 0x7f2404199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 -- 172.21.15.186:0/919801237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2404103e40 msgr2=0x7f2404199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2404103e40 0x7f2404199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.245 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 -- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24041047e0 msgr2=0x7f24041a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24041047e0 0x7f24041a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 -- 172.21.15.186:0/919801237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24041a0a70 con 0x7f2404100dc0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.244+0000 7f24037fe700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b530 tx=0x7f23f400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.245+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4015020 con 0x7f2404100dc0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.245+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23f4011d10 con 0x7f2404100dc0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.245+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400f2e0 con 0x7f2404100dc0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.245+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24041a0d60 con 0x7f2404100dc0 2024-04-20T21:28:02.246 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.245+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a1210 con 0x7f2404100dc0 2024-04-20T21:28:02.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.246+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240402df90 con 0x7f2404100dc0 2024-04-20T21:28:02.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.247+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f23f4018020 con 0x7f2404100dc0 2024-04-20T21:28:02.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.247+0000 7f2400ff9700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 0x7f23ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:02.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.247+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f23f40ceff0 con 0x7f2404100dc0 2024-04-20T21:28:02.247 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.248+0000 7f2402ffd700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 0x7f23ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:02.248 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.249+0000 7f2402ffd700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 0x7f23ec0a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f23f80039c0 tx=0x7f23f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:02.251 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.252+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f23f401f020 con 0x7f2404100dc0 2024-04-20T21:28:02.458 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:02.459+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2404062e90 con 0x7f23ec0a3cc0 2024-04-20T21:28:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:02.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:02 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:02.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:02.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:02.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:02 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:03.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:03.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:03.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:02 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:03.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:03 smithi186 ceph-mon[22157]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:03.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:03 smithi186 ceph-mon[22157]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:03.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:03 smithi186 ceph-mon[22157]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:04.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:03 smithi007 ceph-mon[16867]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:04.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:03 smithi007 ceph-mon[16867]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:04.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:03 smithi007 ceph-mon[16867]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:04.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:03 smithi016 ceph-mon[21409]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:04.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:03 smithi016 ceph-mon[21409]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:04.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:03 smithi016 ceph-mon[21409]: osdmap e55: 8 total, 8 up, 8 in 2024-04-20T21:28:04.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:04 smithi186 ceph-mon[22157]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:05.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:04 smithi007 ceph-mon[16867]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:05.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:04 smithi016 ceph-mon[21409]: osdmap e56: 8 total, 8 up, 8 in 2024-04-20T21:28:05.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/1264600420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]': finished 2024-04-20T21:28:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T21:28:05.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/1264600420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]': finished 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T21:28:06.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:06.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/1264600420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]: dispatch 2024-04-20T21:28:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d25aec8d-214f-44ed-8166-4b0dfde293d5"}]': finished 2024-04-20T21:28:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: osdmap e57: 9 total, 8 up, 9 in 2024-04-20T21:28:06.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:07.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:06 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/2978017272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:07.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:06 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/2978017272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:07.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:06 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/2978017272' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:08.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:07 smithi007 ceph-mon[16867]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:08.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:07 smithi016 ceph-mon[21409]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:08.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:07 smithi186 ceph-mon[22157]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:09.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:09 smithi186 ceph-mon[22157]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:09.997 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:09 smithi016 ceph-mon[21409]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:10.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:09 smithi007 ceph-mon[16867]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:11.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:11 smithi186 ceph-mon[22157]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:12.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:11 smithi007 ceph-mon[16867]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:12.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:11 smithi016 ceph-mon[21409]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:12 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:12 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:13.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:12 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:13.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:12 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:13.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:12 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-20T21:28:13.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:12 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:13 smithi186 ceph-mon[22157]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:13 smithi186 ceph-mon[22157]: Deploying daemon osd.8 on smithi186 2024-04-20T21:28:14.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:13 smithi007 ceph-mon[16867]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:14.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:13 smithi007 ceph-mon[16867]: Deploying daemon osd.8 on smithi186 2024-04-20T21:28:14.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:13 smithi016 ceph-mon[21409]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:14.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:13 smithi016 ceph-mon[21409]: Deploying daemon osd.8 on smithi186 2024-04-20T21:28:15.803 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:15 smithi186 ceph-mon[22157]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:16.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:15 smithi007 ceph-mon[16867]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:16.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:15 smithi016 ceph-mon[21409]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:18.004 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:17 smithi186 ceph-mon[22157]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:18.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:17 smithi007 ceph-mon[16867]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:18.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:17 smithi016 ceph-mon[21409]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:19.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:19 smithi016 ceph-mon[21409]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:19 smithi186 ceph-mon[22157]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:19 smithi007 ceph-mon[16867]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:21.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:21 smithi016 ceph-mon[21409]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:21 smithi186 ceph-mon[22157]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:22.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:21 smithi007 ceph-mon[16867]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:22.763 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 8 on host 'smithi186' 2024-04-20T21:28:22.763 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.763+0000 7f2400ff9700 1 -- 172.21.15.186:0/919801237 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2404062e90 con 0x7f23ec0a3cc0 2024-04-20T21:28:22.765 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 msgr2=0x7f23ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:22.765 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 0x7f23ec0a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f23f80039c0 tx=0x7f23f8005fb0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.765 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 msgr2=0x7f2404198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:22.765 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b530 tx=0x7f23f400b8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.765 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 shutdown_connections 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23ec0a3cc0 0x7f23ec0a6180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2404100dc0 0x7f2404198e10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24041047e0 0x7f24041a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 --2- 172.21.15.186:0/919801237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2404103e40 0x7f2404199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 >> 172.21.15.186:0/919801237 conn(0x7f24040fc720 msgr2=0x7f2404107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 shutdown_connections 2024-04-20T21:28:22.766 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:28:22.766+0000 7f2409ec7700 1 -- 172.21.15.186:0/919801237 wait complete. 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:22 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:23.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:23.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:22 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:22 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:23.466 DEBUG:teuthology.orchestra.run.smithi186:osd.8> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.8.service 2024-04-20T21:28:23.468 INFO:tasks.cephadm:Deploying osd.9 on smithi007 with /dev/vg_nvme/lv_1... 2024-04-20T21:28:23.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:28:23.936 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:23.986 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-mon[22157]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:23.986 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:28:23.887+0000 7f13573d6700 -1 osd.8 0 waiting for initial osdmap 2024-04-20T21:28:23.987 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:28:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:28:23.899+0000 7f13531c9700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:24.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:23 smithi016 ceph-mon[21409]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: from='osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: osdmap e58: 9 total, 8 up, 9 in 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:24.249 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:23 smithi007 ceph-mon[16867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:28:25.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:24 smithi186 ceph-mon[22157]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:28:25.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:24 smithi186 ceph-mon[22157]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:28:25.093 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:24 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:25.094 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:24 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:25.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:24 smithi016 ceph-mon[21409]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:28:25.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:24 smithi016 ceph-mon[21409]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:28:25.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:24 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:25.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:24 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:25.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:24 smithi007 ceph-mon[16867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:28:25.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:24 smithi007 ceph-mon[16867]: osdmap e59: 9 total, 8 up, 9 in 2024-04-20T21:28:25.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:24 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:25.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:24 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:26.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] boot 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:25 smithi016 ceph-mon[21409]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T21:28:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:28:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:28:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] boot 2024-04-20T21:28:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T21:28:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:25 smithi186 ceph-mon[22157]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: osd.8 [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] boot 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: osdmap e60: 9 total, 9 up, 9 in 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:28:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:25 smithi007 ceph-mon[16867]: osdmap e61: 9 total, 9 up, 9 in 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10+0 records in 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:10+0 records out 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0325908 s, 322 MB/s 2024-04-20T21:28:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:--> Zapping successful for: 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi186 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-20T21:28:28.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.035 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi186 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:28:28.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:28.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: pgmap v220: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi186 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 5696M 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: osdmap e62: 9 total, 9 up, 9 in 2024-04-20T21:28:28.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:28.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:28.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:28:28.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:28.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi007:vg_nvme/lv_1 2024-04-20T21:28:28.702 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:28:29.177 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:29 smithi007 ceph-mon[16867]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:28:29.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:29 smithi016 ceph-mon[21409]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:28:29.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:29 smithi186 ceph-mon[22157]: osdmap e63: 9 total, 9 up, 9 in 2024-04-20T21:28:30.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:30 smithi007 ceph-mon[16867]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:30.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:30 smithi007 ceph-mon[16867]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:28:30.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:30 smithi016 ceph-mon[21409]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:30.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:30 smithi016 ceph-mon[21409]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:28:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:30 smithi186 ceph-mon[22157]: pgmap v223: 1 pgs: 1 peering; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:30 smithi186 ceph-mon[22157]: osdmap e64: 9 total, 9 up, 9 in 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 -- 172.21.15.7:0/691174876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c10a8b0 msgr2=0x7f9f1c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 --2- 172.21.15.7:0/691174876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c10cd40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400b0a0 tx=0x7f9f1400b3b0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 -- 172.21.15.7:0/691174876 shutdown_connections 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 --2- 172.21.15.7:0/691174876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c076a80 0x7f9f1c074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 --2- 172.21.15.7:0/691174876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c10cd40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 --2- 172.21.15.7:0/691174876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 -- 172.21.15.7:0/691174876 >> 172.21.15.7:0/691174876 conn(0x7f9f1c0fdf80 msgr2=0x7f9f1c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 -- 172.21.15.7:0/691174876 shutdown_connections 2024-04-20T21:28:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.292+0000 7f9f21600700 1 -- 172.21.15.7:0/691174876 wait complete. 2024-04-20T21:28:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.293+0000 7f9f21600700 1 Processor -- start 2024-04-20T21:28:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.293+0000 7f9f21600700 1 -- start start 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 0x7f9f1c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c110830 con 0x7f9f1c076a80 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c1106b0 con 0x7f9f1c0754b0 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f21600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c1109b0 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f1be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f20c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f1be00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:50868/0 (socket says 172.21.15.7:50868) 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f1be00700 1 -- 172.21.15.7:0/1157520126 learned_addr learned my addr 172.21.15.7:0/1157520126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:28:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f13400700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 0x7f9f1c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f20c00700 1 -- 172.21.15.7:0/1157520126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 msgr2=0x7f9f1c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f20c00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f20c00700 1 -- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 msgr2=0x7f9f1c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.294+0000 7f9f20c00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 0x7f9f1c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f20c00700 1 -- 172.21.15.7:0/1157520126 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c1a29f0 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f1be00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f13400700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 0x7f9f1c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f20c00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c1a2320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400b070 tx=0x7f9f14004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14021070 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f14004e50 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c1a2c80 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f140047c0 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.295+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c1a3130 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.296+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c069740 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.298+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9f14003a00 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.300+0000 7f9f19600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 0x7f9f000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.300+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f9f1400f020 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.300+0000 7f9f1be00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 0x7f9f000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.301+0000 7f9f1be00700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 0x7f9f000a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c19c280 tx=0x7f9f0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:31.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.302+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9f1402a020 con 0x7f9f1c10a8b0 2024-04-20T21:28:31.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:31.548+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9f1c062e90 con 0x7f9f000a3c40 2024-04-20T21:28:32.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:32 smithi016 ceph-mon[21409]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-20T21:28:32.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:32.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:32.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:32 smithi186 ceph-mon[22157]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-20T21:28:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:32.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:32 smithi007 ceph-mon[16867]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-20T21:28:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:28:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:28:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:33.375 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:33 smithi007 ceph-mon[16867]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:33.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:33 smithi016 ceph-mon[21409]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:33.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:33 smithi186 ceph-mon[22157]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi007:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:28:34.372 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:34 smithi007 ceph-mon[16867]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:28:34.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:34 smithi016 ceph-mon[21409]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:28:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:34 smithi186 ceph-mon[22157]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-20T21:28:36.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-20T21:28:36.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1953154919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.366 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]': finished 2024-04-20T21:28:36.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T21:28:36.367 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:28:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-20T21:28:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1953154919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]': finished 2024-04-20T21:28:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T21:28:36.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:28:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-20T21:28:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1953154919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]: dispatch 2024-04-20T21:28:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4"}]': finished 2024-04-20T21:28:36.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: osdmap e65: 10 total, 9 up, 10 in 2024-04-20T21:28:36.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:28:37.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1537936762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:37.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1537936762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:37.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1537936762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:28:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:38 smithi016 ceph-mon[21409]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:28:38.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:38 smithi186 ceph-mon[22157]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:28:38.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:38 smithi007 ceph-mon[16867]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-20T21:28:40.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:40 smithi007 ceph-mon[16867]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-20T21:28:40.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:40 smithi016 ceph-mon[21409]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-20T21:28:40.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:40 smithi186 ceph-mon[22157]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-20T21:28:42.331 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:42 smithi007 ceph-mon[16867]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:42 smithi016 ceph-mon[21409]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:42 smithi186 ceph-mon[22157]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:44 smithi016 ceph-mon[21409]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:44.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:44 smithi186 ceph-mon[22157]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:44.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:44 smithi007 ceph-mon[16867]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:46 smithi016 ceph-mon[21409]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:46.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:28:46.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:46 smithi186 ceph-mon[22157]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:28:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:46.486 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:46 smithi007 ceph-mon[16867]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:46.486 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-20T21:28:46.486 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:28:47.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:47 smithi016 ceph-mon[21409]: Deploying daemon osd.9 on smithi007 2024-04-20T21:28:47.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:47 smithi186 ceph-mon[22157]: Deploying daemon osd.9 on smithi007 2024-04-20T21:28:47.514 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:47 smithi007 ceph-mon[16867]: Deploying daemon osd.9 on smithi007 2024-04-20T21:28:48.380 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:48 smithi007 ceph-mon[16867]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:48.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:48 smithi016 ceph-mon[21409]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:48.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:48 smithi186 ceph-mon[22157]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:50.223 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:50 smithi007 ceph-mon[16867]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:50.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:50 smithi016 ceph-mon[21409]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:50 smithi186 ceph-mon[22157]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:52.215 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:52 smithi007 ceph-mon[16867]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:52.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:52 smithi016 ceph-mon[21409]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:52.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:52 smithi186 ceph-mon[22157]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:54.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:54 smithi016 ceph-mon[21409]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:54 smithi186 ceph-mon[22157]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:54.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:54.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:54 smithi007 ceph-mon[16867]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:54.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:54.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:28:54.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:56.383 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:56 smithi007 ceph-mon[16867]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:56.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:56 smithi016 ceph-mon[21409]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:56.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:56 smithi186 ceph-mon[22157]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:58 smithi007 ceph-mon[16867]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:58 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:58 smithi007 ceph-mon[16867]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:28:58.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:58 smithi016 ceph-mon[21409]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:58 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:58 smithi016 ceph-mon[21409]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:28:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:58 smithi186 ceph-mon[22157]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:28:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:58 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:28:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:58 smithi186 ceph-mon[22157]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-20T21:28:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.887+0000 7f9f19600700 1 -- 172.21.15.7:0/1157520126 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f1c062e90 con 0x7f9f000a3c40 2024-04-20T21:28:58.887 INFO:teuthology.orchestra.run.smithi007.stdout:Created osd(s) 9 on host 'smithi007' 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.890+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 msgr2=0x7f9f000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.890+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 0x7f9f000a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c19c280 tx=0x7f9f0c009260 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 msgr2=0x7f9f1c1a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c1a2320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400b070 tx=0x7f9f14004c60 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 shutdown_connections 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9f000a3c40 0x7f9f000a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f1c10a8b0 0x7f9f1c1a2320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c0754b0 0x7f9f1c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 --2- 172.21.15.7:0/1157520126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f1c076a80 0x7f9f1c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 >> 172.21.15.7:0/1157520126 conn(0x7f9f1c0fdf80 msgr2=0x7f9f1c1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 shutdown_connections 2024-04-20T21:28:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:28:58.891+0000 7f9f21600700 1 -- 172.21.15.7:0/1157520126 wait complete. 2024-04-20T21:28:59.714 DEBUG:teuthology.orchestra.run.smithi007:osd.9> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.9.service 2024-04-20T21:28:59.721 INFO:tasks.cephadm:Deploying osd.10 on smithi016 with /dev/vg_nvme/lv_1... 2024-04-20T21:28:59.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:28:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:29:00.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:00.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:00.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T21:29:00.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:29:00.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:00.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:28:59 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.103 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:28:59 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: osdmap e66: 10 total, 9 up, 10 in 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]: dispatch 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:28:59 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:00.664 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:29:00.450+0000 7f46eea00700 -1 osd.9 0 waiting for initial osdmap 2024-04-20T21:29:00.664 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:29:00.486+0000 7f46e3e00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:29:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-mon[16867]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:29:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-mon[16867]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:00 smithi007 ceph-mon[16867]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' 2024-04-20T21:29:01.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:00 smithi016 ceph-mon[21409]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:29:01.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:00 smithi016 ceph-mon[21409]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:01.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:00 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:00 smithi016 ceph-mon[21409]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' 2024-04-20T21:29:01.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:00 smithi186 ceph-mon[22157]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi007", "root=default"]}]': finished 2024-04-20T21:29:01.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:00 smithi186 ceph-mon[22157]: osdmap e67: 10 total, 9 up, 10 in 2024-04-20T21:29:01.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:00 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:00 smithi186 ceph-mon[22157]: from='osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642]' entity='osd.9' 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] boot 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T21:29:01.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:01 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:02.062 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] boot 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T21:29:02.063 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:01 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10+0 records in 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:10+0 records out 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00952414 s, 1.1 GB/s 2024-04-20T21:29:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:--> Zapping successful for: 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: osd.9 [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] boot 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: osdmap e68: 10 total, 10 up, 10 in 2024-04-20T21:29:02.117 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:01 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:29:02.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-04-20T21:29:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:29:04.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:03 smithi007 ceph-mon[16867]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:04.092 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:03 smithi007 ceph-mon[16867]: osdmap e69: 10 total, 10 up, 10 in 2024-04-20T21:29:04.144 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:03 smithi016 ceph-mon[21409]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:04.144 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:03 smithi016 ceph-mon[21409]: osdmap e69: 10 total, 10 up, 10 in 2024-04-20T21:29:04.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:03 smithi186 ceph-mon[22157]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:04.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:03 smithi186 ceph-mon[22157]: osdmap e69: 10 total, 10 up, 10 in 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 -- 172.21.15.16:0/3824691997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c102ab0 msgr2=0x7fd88c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 --2- 172.21.15.16:0/3824691997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c102ab0 0x7fd88c102ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00b0d0 tx=0x7fd87c00b3e0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 -- 172.21.15.16:0/3824691997 shutdown_connections 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 --2- 172.21.15.16:0/3824691997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c104650 0x7fd88c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 --2- 172.21.15.16:0/3824691997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c102ab0 0x7fd88c102ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 --2- 172.21.15.16:0/3824691997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c103cb0 0x7fd88c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 -- 172.21.15.16:0/3824691997 >> 172.21.15.16:0/3824691997 conn(0x7fd88c0fe050 msgr2=0x7fd88c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 -- 172.21.15.16:0/3824691997 shutdown_connections 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.676+0000 7fd89347f700 1 -- 172.21.15.16:0/3824691997 wait complete. 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 Processor -- start 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 -- start start 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102ab0 0x7fd88c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c103cb0 0x7fd88c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10e710 con 0x7fd88c103cb0 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10e590 con 0x7fd88c104650 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89347f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c10e890 con 0x7fd88c102ab0 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50032/0 (socket says 172.21.15.16:50032) 2024-04-20T21:29:04.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 -- 172.21.15.16:0/908673554 learned_addr learned my addr 172.21.15.16:0/908673554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd89121b700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102ab0 0x7fd88c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 -- 172.21.15.16:0/908673554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102ab0 msgr2=0x7fd88c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102ab0 0x7fd88c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 -- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c103cb0 msgr2=0x7fd88c1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c103cb0 0x7fd88c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.677+0000 7fd891a1c700 1 -- 172.21.15.16:0/908673554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c1a09e0 con 0x7fd88c104650 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd891a1c700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b550 tx=0x7fd88400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884015020 con 0x7fd88c104650 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884011d10 con 0x7fd88c104650 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c1a0cd0 con 0x7fd88c104650 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88400f2e0 con 0x7fd88c104650 2024-04-20T21:29:04.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.678+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1a1280 con 0x7fd88c104650 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.679+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c02df90 con 0x7fd88c104650 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.680+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fd884018020 con 0x7fd88c104650 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.681+0000 7fd88a7fc700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 0x7fd8740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.681+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd8840cf800 con 0x7fd88c104650 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.681+0000 7fd89121b700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 0x7fd8740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.682+0000 7fd89121b700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 0x7fd8740a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd88c19a270 tx=0x7fd87c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:04.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.685+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd88401f020 con 0x7fd88c104650 2024-04-20T21:29:04.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:04.904+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd88c062e90 con 0x7fd8740a3cc0 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi007 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:05.830 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 4273M 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:05.831 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:05 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi007 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 4273M 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:06.217 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:05 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:06.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi007 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 4273M 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:06.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:06.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:06.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:06.285 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:05 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/978427809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]': finished 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T21:29:08.142 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:08.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:08.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/978427809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]': finished 2024-04-20T21:29:08.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T21:29:08.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/978427809' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]: dispatch 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "98497028-afaa-4bae-a644-e31beddd7acd"}]': finished 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: osdmap e70: 11 total, 10 up, 11 in 2024-04-20T21:29:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:09.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.16:0/2229503067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:09.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.16:0/2229503067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:09.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.16:0/2229503067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:10.096 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:09 smithi016 ceph-mon[21409]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:10.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:09 smithi186 ceph-mon[22157]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:10.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:09 smithi007 ceph-mon[16867]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:12.190 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:11 smithi016 ceph-mon[21409]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:12.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:11 smithi186 ceph-mon[22157]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:12.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:11 smithi007 ceph-mon[16867]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:14.122 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:13 smithi016 ceph-mon[21409]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:14.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:13 smithi186 ceph-mon[22157]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:14.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:13 smithi007 ceph-mon[16867]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:16.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:15 smithi016 ceph-mon[21409]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:16.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:16.141 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:15 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:16.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:15 smithi186 ceph-mon[22157]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:16.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:16.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:15 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:16.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:15 smithi007 ceph-mon[16867]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:16.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-20T21:29:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:15 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:17.131 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:16 smithi016 ceph-mon[21409]: Deploying daemon osd.10 on smithi016 2024-04-20T21:29:17.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:16 smithi186 ceph-mon[22157]: Deploying daemon osd.10 on smithi016 2024-04-20T21:29:17.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:16 smithi007 ceph-mon[16867]: Deploying daemon osd.10 on smithi016 2024-04-20T21:29:18.171 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:17 smithi016 ceph-mon[21409]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:18.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:17 smithi186 ceph-mon[22157]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:18.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:17 smithi007 ceph-mon[16867]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:20.113 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:19 smithi007 ceph-mon[16867]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:20.139 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:19 smithi016 ceph-mon[21409]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:20.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:19 smithi186 ceph-mon[22157]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:22.046 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:21 smithi016 ceph-mon[21409]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:22.046 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.046 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.046 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:22.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:21 smithi186 ceph-mon[22157]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:22.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:22.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:21 smithi007 ceph-mon[16867]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:22.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:22.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:23.934 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:23 smithi016 ceph-mon[21409]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:24.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:23 smithi186 ceph-mon[22157]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:24.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:23 smithi007 ceph-mon[16867]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:25.094 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 10 on host 'smithi016' 2024-04-20T21:29:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.094+0000 7fd88a7fc700 1 -- 172.21.15.16:0/908673554 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd88c062e90 con 0x7fd8740a3cc0 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 msgr2=0x7fd8740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 0x7fd8740a6180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd88c19a270 tx=0x7fd87c00b040 comp rx=0 tx=0).stop 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 msgr2=0x7fd88c1a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b550 tx=0x7fd88400b860 comp rx=0 tx=0).stop 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 shutdown_connections 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd8740a3cc0 0x7fd8740a6180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102ab0 0x7fd88c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88c104650 0x7fd88c1a0310 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 --2- 172.21.15.16:0/908673554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd88c103cb0 0x7fd88c1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 >> 172.21.15.16:0/908673554 conn(0x7fd88c0fe050 msgr2=0x7fd88c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 shutdown_connections 2024-04-20T21:29:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-20T21:29:25.096+0000 7fd89347f700 1 -- 172.21.15.16:0/908673554 wait complete. 2024-04-20T21:29:25.898 DEBUG:teuthology.orchestra.run.smithi016:osd.10> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.10.service 2024-04-20T21:29:25.900 INFO:tasks.cephadm:Deploying osd.11 on smithi186 with /dev/vg_nvme/lv_1... 2024-04-20T21:29:25.900 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-20T21:29:26.152 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:29:26.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:26 smithi016 ceph-mon[21409]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:26 smithi016 ceph-mon[21409]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:26.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:26 smithi016 ceph-mon[21409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:26 smithi186 ceph-mon[22157]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:26.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:26 smithi186 ceph-mon[22157]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:26.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:26 smithi186 ceph-mon[22157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:26 smithi007 ceph-mon[16867]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:26 smithi007 ceph-mon[16867]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:26 smithi007 ceph-mon[16867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-20T21:29:27.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.337 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:29:27.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T21:29:27.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:27.338 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:27 smithi186 ceph-mon[22157]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:29:27.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T21:29:27.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:27.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-mon[21409]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: osdmap e71: 11 total, 10 up, 11 in 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:27.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:27 smithi007 ceph-mon[16867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-04-20T21:29:27.889 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:29:27.589+0000 7f8bf24a2700 -1 osd.10 0 waiting for initial osdmap 2024-04-20T21:29:27.889 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:29:27 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:29:27.603+0000 7f8bea28d700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:29:28.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:29:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:28 smithi016 ceph-mon[21409]: from='osd.10 ' entity='osd.10' 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011443 s, 916 MB/s 2024-04-20T21:29:28.246 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2024-04-20T21:29:28.246 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:28.246 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:29:28.246 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:28.246 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.246 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.247 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:28 smithi186 ceph-mon[22157]: from='osd.10 ' entity='osd.10' 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: osdmap e72: 11 total, 10 up, 11 in 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:28 smithi007 ceph-mon[16867]: from='osd.10 ' entity='osd.10' 2024-04-20T21:29:28.924 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2024-04-20T21:29:29.165 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:29:29.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:29:29.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:29:29.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] boot 2024-04-20T21:29:29.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T21:29:29.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:29.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:29 smithi186 ceph-mon[22157]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T21:29:29.363 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:29:29.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:29:29.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] boot 2024-04-20T21:29:29.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T21:29:29.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:29.364 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:29 smithi016 ceph-mon[21409]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: osd.10 [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] boot 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: osdmap e73: 11 total, 11 up, 11 in 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:29:29.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:29 smithi007 ceph-mon[16867]: osdmap e74: 11 total, 11 up, 11 in 2024-04-20T21:29:30.330 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:30 smithi186 ceph-mon[22157]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:30.330 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:30 smithi186 ceph-mon[22157]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:29:30.350 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:30 smithi016 ceph-mon[21409]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:30.350 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:30 smithi016 ceph-mon[21409]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:29:30.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:30 smithi007 ceph-mon[16867]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 283 MiB used, 894 GiB / 894 GiB avail 2024-04-20T21:29:30.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:30 smithi007 ceph-mon[16867]: osdmap e75: 11 total, 11 up, 11 in 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.856+0000 7f3dab681700 1 -- 172.21.15.186:0/723030235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da4100d60 msgr2=0x7f3da4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.856+0000 7f3dab681700 1 --2- 172.21.15.186:0/723030235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da4100d60 0x7f3da4101180 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009990 tx=0x7f3da0009ca0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- 172.21.15.186:0/723030235 shutdown_connections 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- 172.21.15.186:0/723030235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da4100d60 0x7f3da4101180 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- 172.21.15.186:0/723030235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da41046e0 0x7f3da410af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- 172.21.15.186:0/723030235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 0x7f3da41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- 172.21.15.186:0/723030235 >> 172.21.15.186:0/723030235 conn(0x7f3da40fc6e0 msgr2=0x7f3da40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- 172.21.15.186:0/723030235 shutdown_connections 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- 172.21.15.186:0/723030235 wait complete. 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 Processor -- start 2024-04-20T21:29:30.858 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- start start 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4100d60 0x7f3da4198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 0x7f3da4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410e6f0 con 0x7f3da4100d60 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410e570 con 0x7f3da4103d40 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.857+0000 7f3dab681700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3da410e870 con 0x7f3da41046e0 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47698/0 (socket says 172.21.15.186:47698) 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 -- 172.21.15.186:0/2496911343 learned_addr learned my addr 172.21.15.186:0/2496911343 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da8c1c700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 0x7f3da4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da941d700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4100d60 0x7f3da4198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 -- 172.21.15.186:0/2496911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 msgr2=0x7f3da4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 0x7f3da4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.859 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 -- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4100d60 msgr2=0x7f3da4198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4100d60 0x7f3da4198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 -- 172.21.15.186:0/2496911343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da41a0990 con 0x7f3da41046e0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3da9c1e700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00b9a0 tx=0x7f3d9c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c019070 con 0x7f3da41046e0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da41a0c80 con 0x7f3da41046e0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d9c00ce10 con 0x7f3da41046e0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0073d0 con 0x7f3da41046e0 2024-04-20T21:29:30.860 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.858+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da41a1170 con 0x7f3da41046e0 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.859+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da4069740 con 0x7f3da41046e0 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.860+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f3d9c01c070 con 0x7f3da41046e0 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.861+0000 7f3d9a7fc700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 0x7f3d900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.861+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f3d9c0cf110 con 0x7f3da41046e0 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.861+0000 7f3da941d700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 0x7f3d900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.861 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.862+0000 7f3da941d700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 0x7f3d900a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3da00039c0 tx=0x7f3da0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.864 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:30.866+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3d9c09c140 con 0x7f3da41046e0 2024-04-20T21:29:31.080 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:31.081+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3da4062e90 con 0x7f3d900a3c40 2024-04-20T21:29:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: pgmap v266: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:29:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi016 2024-04-20T21:29:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 4273M 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:31 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: pgmap v266: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi016 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 4273M 2024-04-20T21:29:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:31 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: pgmap v266: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: osdmap e76: 11 total, 11 up, 11 in 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi016 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 4273M 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:29:31.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-20T21:29:31.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-20T21:29:31.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:31.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:31 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:32 smithi016 ceph-mon[21409]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:29:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:32 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:32 smithi186 ceph-mon[22157]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:29:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:32 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:32 smithi007 ceph-mon[16867]: osdmap e77: 11 total, 11 up, 11 in 2024-04-20T21:29:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:32 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:34.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:33 smithi016 ceph-mon[21409]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:34.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:33 smithi186 ceph-mon[22157]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:34.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:33 smithi007 ceph-mon[16867]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/2547338900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]': finished 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:35.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:34 smithi016 ceph-mon[21409]: from='client.? 172.21.15.186:0/3316855661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:35.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/2547338900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]': finished 2024-04-20T21:29:35.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T21:29:35.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:35.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:34 smithi186 ceph-mon[22157]: from='client.? 172.21.15.186:0/3316855661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:35.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/2547338900' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]: dispatch 2024-04-20T21:29:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d80cce8-3e29-4afe-b398-5388ebfe1b5c"}]': finished 2024-04-20T21:29:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: osdmap e78: 12 total, 11 up, 12 in 2024-04-20T21:29:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.186:0/3316855661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-20T21:29:36.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:35 smithi016 ceph-mon[21409]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-20T21:29:36.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:35 smithi186 ceph-mon[22157]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-20T21:29:36.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:35 smithi007 ceph-mon[16867]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-20T21:29:38.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:37 smithi016 ceph-mon[21409]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:38.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:37 smithi186 ceph-mon[22157]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:38.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:37 smithi007 ceph-mon[16867]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:40.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:39 smithi016 ceph-mon[21409]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:40.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:39 smithi186 ceph-mon[22157]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:40.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:39 smithi007 ceph-mon[16867]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:42.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:41 smithi016 ceph-mon[21409]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:42.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:41 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:29:42.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:41 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:41 smithi186 ceph-mon[22157]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:42.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:41 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:29:42.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:41 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:42.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:41 smithi007 ceph-mon[16867]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:41 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-20T21:29:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:41 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:43.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:42 smithi186 ceph-mon[22157]: Deploying daemon osd.11 on smithi186 2024-04-20T21:29:43.249 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:42 smithi016 ceph-mon[21409]: Deploying daemon osd.11 on smithi186 2024-04-20T21:29:43.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:42 smithi007 ceph-mon[16867]: Deploying daemon osd.11 on smithi186 2024-04-20T21:29:44.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:43 smithi186 ceph-mon[22157]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:44.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:43 smithi007 ceph-mon[16867]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:43 smithi016 ceph-mon[21409]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:46.181 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:45 smithi186 ceph-mon[22157]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:46.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:45 smithi007 ceph-mon[16867]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:45 smithi016 ceph-mon[21409]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:48.179 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:47 smithi186 ceph-mon[22157]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:48.180 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.180 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.180 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:47 smithi007 ceph-mon[16867]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:48.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:48.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:47 smithi016 ceph-mon[21409]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:48.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:48.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:50.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:49 smithi007 ceph-mon[16867]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:50.290 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:49 smithi186 ceph-mon[22157]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:50.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:49 smithi016 ceph-mon[21409]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:51.634 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 11 on host 'smithi186' 2024-04-20T21:29:51.634 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.634+0000 7f3d9a7fc700 1 -- 172.21.15.186:0/2496911343 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3da4062e90 con 0x7f3d900a3c40 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 msgr2=0x7f3d900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 0x7f3d900a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3da00039c0 tx=0x7f3da0005fb0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 msgr2=0x7f3da41a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00b9a0 tx=0x7f3d9c00bcb0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 shutdown_connections 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3d900a3c40 0x7f3d900a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3da41046e0 0x7f3da41a02f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3da4100d60 0x7f3da4198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.636 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 --2- 172.21.15.186:0/2496911343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da4103d40 0x7f3da4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 >> 172.21.15.186:0/2496911343 conn(0x7f3da40fc6e0 msgr2=0x7f3da4107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:51.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 shutdown_connections 2024-04-20T21:29:51.637 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:29:51.637+0000 7f3dab681700 1 -- 172.21.15.186:0/2496911343 wait complete. 2024-04-20T21:29:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:51 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:52.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:51 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:52.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:52.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:51 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:29:52.371 DEBUG:teuthology.orchestra.run.smithi186:osd.11> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.11.service 2024-04-20T21:29:52.374 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-20T21:29:52.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-20T21:29:52.834 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:29:53.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-mon[22157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:29:53.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-mon[22157]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T21:29:53.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:53.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-mon[22157]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:53.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-mon[22157]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:53.216 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:29:52 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:29:52.988+0000 7fb910d77700 -1 osd.11 0 waiting for initial osdmap 2024-04-20T21:29:53.216 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:29:53 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:29:53.001+0000 7fb90cb6a700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:29:53.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:52 smithi007 ceph-mon[16867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:29:53.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:52 smithi007 ceph-mon[16867]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T21:29:53.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:53.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:52 smithi007 ceph-mon[16867]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:53.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:52 smithi007 ceph-mon[16867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:53.322 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:52 smithi016 ceph-mon[21409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-20T21:29:53.322 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:52 smithi016 ceph-mon[21409]: osdmap e79: 12 total, 11 up, 12 in 2024-04-20T21:29:53.322 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:53.322 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:52 smithi016 ceph-mon[21409]: from='osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:53.322 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:52 smithi016 ceph-mon[21409]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2024-04-20T21:29:54.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:54 smithi007 ceph-mon[16867]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:54.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:54 smithi007 ceph-mon[16867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:29:54.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:54 smithi007 ceph-mon[16867]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:29:54.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:54.279 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:54.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:54 smithi016 ceph-mon[21409]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:54 smithi016 ceph-mon[21409]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:29:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:54 smithi016 ceph-mon[21409]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:29:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:54.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:54 smithi186 ceph-mon[22157]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:54 smithi186 ceph-mon[22157]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2024-04-20T21:29:54.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:54 smithi186 ceph-mon[22157]: osdmap e80: 12 total, 11 up, 12 in 2024-04-20T21:29:54.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:54.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 -- 172.21.15.7:0/1615745395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0754b0 msgr2=0x7f9a0c075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 --2- 172.21.15.7:0/1615745395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c075930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009990 tx=0x7f99f4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 -- 172.21.15.7:0/1615745395 shutdown_connections 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 --2- 172.21.15.7:0/1615745395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 --2- 172.21.15.7:0/1615745395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 --2- 172.21.15.7:0/1615745395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c075930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.091+0000 7f9a0b200700 1 -- 172.21.15.7:0/1615745395 >> 172.21.15.7:0/1615745395 conn(0x7f9a0c0fdf80 msgr2=0x7f9a0c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.092+0000 7f9a0b200700 1 -- 172.21.15.7:0/1615745395 shutdown_connections 2024-04-20T21:29:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.092+0000 7f9a0b200700 1 -- 172.21.15.7:0/1615745395 wait complete. 2024-04-20T21:29:55.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 Processor -- start 2024-04-20T21:29:55.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 -- start start 2024-04-20T21:29:55.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c110830 con 0x7f9a0c0754b0 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1106b0 con 0x7f9a0c076a80 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1109b0 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0a800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0a800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43636/0 (socket says 172.21.15.7:43636) 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a0a800700 1 -- 172.21.15.7:0/993120324 learned_addr learned my addr 172.21.15.7:0/993120324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.093+0000 7f9a09e00700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a01400700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 -- 172.21.15.7:0/993120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 msgr2=0x7f9a0c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 -- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c0754b0 msgr2=0x7f9a0c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 -- 172.21.15.7:0/993120324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c1a2a80 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a01400700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0a800700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d9b0 tx=0x7f9a0400dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04017070 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.094+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c1a2d70 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.095+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a04009e20 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.095+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04005410 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.095+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c1a3270 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.096+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c069740 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.098+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9a040055b0 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.099+0000 7f9a03400700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 0x7f99f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.099+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9a040cfe50 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.099+0000 7f9a09e00700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 0x7f99f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.100+0000 7f9a09e00700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 0x7f99f00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c076e60 tx=0x7f99fc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.103+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a04020020 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: purged_snaps scrub starts 2024-04-20T21:29:55.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: purged_snaps scrub ok 2024-04-20T21:29:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] boot 2024-04-20T21:29:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T21:29:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: purged_snaps scrub starts 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: purged_snaps scrub ok 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] boot 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T21:29:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.336+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9a0c063660 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.337+0000 7f9a03400700 1 -- 172.21.15.7:0/993120324 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f9a0409cc30 con 0x7f9a0c10a8b0 2024-04-20T21:29:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 msgr2=0x7f99f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 0x7f99f00a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c076e60 tx=0x7f99fc00b040 comp rx=0 tx=0).stop 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 msgr2=0x7f9a0c1a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400d9b0 tx=0x7f9a0400dd70 comp rx=0 tx=0).stop 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 shutdown_connections 2024-04-20T21:29:55.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99f00a3c40 0x7f99f00a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a0c10a8b0 0x7f9a0c1a23e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0c0754b0 0x7f9a0c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 --2- 172.21.15.7:0/993120324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a0c076a80 0x7f9a0c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 >> 172.21.15.7:0/993120324 conn(0x7f9a0c0fdf80 msgr2=0x7f9a0c1067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 shutdown_connections 2024-04-20T21:29:55.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:55.340+0000 7f9a0b200700 1 -- 172.21.15.7:0/993120324 wait complete. 2024-04-20T21:29:55.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: purged_snaps scrub starts 2024-04-20T21:29:55.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: purged_snaps scrub ok 2024-04-20T21:29:55.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:55.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: osd.11 [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] boot 2024-04-20T21:29:55.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: osdmap e81: 12 total, 12 up, 12 in 2024-04-20T21:29:55.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:29:56.134 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713648593,"num_in_osds":12,"osd_in_since":1713648573,"num_remapped_pgs":0} 2024-04-20T21:29:56.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/993120324' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:56.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:56 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/993120324' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:56.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:56 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: osdmap e82: 12 total, 12 up, 12 in 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/993120324' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:29:56.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:29:56.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:29:56.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:56 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:29:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:29:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:57 smithi007 ceph-mon[16867]: Detected new or changed devices on smithi186 2024-04-20T21:29:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:57 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-20T21:29:57.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:57 smithi016 ceph-mon[21409]: Detected new or changed devices on smithi186 2024-04-20T21:29:57.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:57 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-20T21:29:57.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:57 smithi186 ceph-mon[22157]: Detected new or changed devices on smithi186 2024-04-20T21:29:57.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:57 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 4272M 2024-04-20T21:29:58.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:58 smithi007 ceph-mon[16867]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:29:58.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:58 smithi016 ceph-mon[21409]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:29:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:58 smithi186 ceph-mon[22157]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.693+0000 7fb6e3400700 1 -- 172.21.15.7:0/851168194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 msgr2=0x7fb6dc1118f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.693+0000 7fb6e3400700 1 --2- 172.21.15.7:0/851168194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc1118f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009f90 tx=0x7fb6d400b330 comp rx=0 tx=0).stop 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 -- 172.21.15.7:0/851168194 shutdown_connections 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 --2- 172.21.15.7:0/851168194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc1118f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 --2- 172.21.15.7:0/851168194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06a760 0x7fb6dc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 --2- 172.21.15.7:0/851168194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06b080 0x7fb6dc10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 -- 172.21.15.7:0/851168194 >> 172.21.15.7:0/851168194 conn(0x7fb6dc078900 msgr2=0x7fb6dc078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 -- 172.21.15.7:0/851168194 shutdown_connections 2024-04-20T21:29:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.694+0000 7fb6e3400700 1 -- 172.21.15.7:0/851168194 wait complete. 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 Processor -- start 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 -- start start 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06a760 0x7fb6dc19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06b080 0x7fb6dc19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc114fc0 con 0x7fb6dc06b080 2024-04-20T21:29:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc114e40 con 0x7fb6dc10f500 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.695+0000 7fb6e3400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc115140 con 0x7fb6dc06a760 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6dbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06b080 0x7fb6dc19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47724/0 (socket says 172.21.15.7:47724) 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 -- 172.21.15.7:0/2630561891 learned_addr learned my addr 172.21.15.7:0/2630561891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06a760 0x7fb6dc19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 -- 172.21.15.7:0/2630561891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06a760 msgr2=0x7fb6dc19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06a760 0x7fb6dc19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 -- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06b080 msgr2=0x7fb6dc19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06b080 0x7fb6dc19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.696+0000 7fb6e1400700 1 -- 172.21.15.7:0/2630561891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc1a0460 con 0x7fb6dc10f500 2024-04-20T21:29:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6e1400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400b6c0 tx=0x7fb6d400cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4021070 con 0x7fb6dc10f500 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4007ac0 con 0x7fb6dc10f500 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d401de60 con 0x7fb6dc10f500 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc1a06f0 con 0x7fb6dc10f500 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.697+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc1a7fd0 con 0x7fb6dc10f500 2024-04-20T21:29:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.698+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc069740 con 0x7fb6dc10f500 2024-04-20T21:29:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.701+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fb6d4024070 con 0x7fb6dc10f500 2024-04-20T21:29:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.702+0000 7fb6d9600700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.703+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6d400f020 con 0x7fb6dc10f500 2024-04-20T21:29:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.703+0000 7fb6e0a00700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.703+0000 7fb6e0a00700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.704+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb6d402b020 con 0x7fb6dc10f500 2024-04-20T21:29:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.940+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6dc02df90 con 0x7fb6dc10f500 2024-04-20T21:29:58.941 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:29:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.941+0000 7fb6d9600700 1 -- 172.21.15.7:0/2630561891 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18340 (secure 0 0 0) 0x7fb6d409e620 con 0x7fb6dc10f500 2024-04-20T21:29:58.942 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:29:55.003374+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.945+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 msgr2=0x7fb6c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.945+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d0009310 comp rx=0 tx=0).stop 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.945+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 msgr2=0x7fb6dc19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.945+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400b6c0 tx=0x7fb6d400cfa0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 shutdown_connections 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb6c40a3cc0 0x7fb6c40a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb6dc06a760 0x7fb6dc19b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6dc10f500 0x7fb6dc19fd20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 --2- 172.21.15.7:0/2630561891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6dc06b080 0x7fb6dc19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 >> 172.21.15.7:0/2630561891 conn(0x7fb6dc078900 msgr2=0x7fb6dc110970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 shutdown_connections 2024-04-20T21:29:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:29:58.946+0000 7fb6e3400700 1 -- 172.21.15.7:0/2630561891 wait complete. 2024-04-20T21:29:59.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:29:59 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2630561891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:29:59.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:29:59 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2630561891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:29:59.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:29:59 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2630561891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:29:59.736 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:59.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-20T21:30:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:30:00.337 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:00 smithi007 ceph-mon[16867]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:00.337 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:00 smithi007 ceph-mon[16867]: overall HEALTH_OK 2024-04-20T21:30:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:00 smithi016 ceph-mon[21409]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:00 smithi016 ceph-mon[21409]: overall HEALTH_OK 2024-04-20T21:30:00.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:00 smithi186 ceph-mon[22157]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:00.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:00 smithi186 ceph-mon[22157]: overall HEALTH_OK 2024-04-20T21:30:02.182 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:02 smithi007 ceph-mon[16867]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:02.287 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:02 smithi186 ceph-mon[22157]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.360+0000 7f2ccc000700 1 -- 172.21.15.7:0/1337951667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 msgr2=0x7f2cc4111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.360+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1337951667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc4111b80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009fc0 tx=0x7f2cbc00b330 comp rx=0 tx=0).stop 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 -- 172.21.15.7:0/1337951667 shutdown_connections 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1337951667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc4111b80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1337951667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc41052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1337951667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 0x7f2cc41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 -- 172.21.15.7:0/1337951667 >> 172.21.15.7:0/1337951667 conn(0x7f2cc4100760 msgr2=0x7f2cc4102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 -- 172.21.15.7:0/1337951667 shutdown_connections 2024-04-20T21:30:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.361+0000 7f2ccc000700 1 -- 172.21.15.7:0/1337951667 wait complete. 2024-04-20T21:30:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.362+0000 7f2ccc000700 1 Processor -- start 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.362+0000 7f2ccc000700 1 -- start start 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 0x7f2cc419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4114fe0 con 0x7f2cc4104ed0 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4114e60 con 0x7f2cc4105880 2024-04-20T21:30:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2ccc000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc4115160 con 0x7f2cc4109fb0 2024-04-20T21:30:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2cc9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2cca000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2cc9600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39664/0 (socket says 172.21.15.7:39664) 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2cc9600700 1 -- 172.21.15.7:0/1962169049 learned_addr learned my addr 172.21.15.7:0/1962169049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.363+0000 7f2cc8c00700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 0x7f2cc419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 -- 172.21.15.7:0/1962169049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 msgr2=0x7f2cc419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 0x7f2cc419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 -- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 msgr2=0x7f2cc419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 -- 172.21.15.7:0/1962169049 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc41a03c0 con 0x7f2cc4109fb0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cca000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc419fc80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc015000 tx=0x7f2cbc00cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc021070 con 0x7f2cc4109fb0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc007ac0 con 0x7f2cc4109fb0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0052a0 con 0x7f2cc4109fb0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.364+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc41a0650 con 0x7f2cc4109fb0 2024-04-20T21:30:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.365+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc41a7f30 con 0x7f2cc4109fb0 2024-04-20T21:30:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.366+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc4069740 con 0x7f2cc4109fb0 2024-04-20T21:30:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.368+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2cbc024070 con 0x7f2cc4109fb0 2024-04-20T21:30:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.369+0000 7f2cc2000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 0x7f2cac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.369+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2cbc0d0ec0 con 0x7f2cc4109fb0 2024-04-20T21:30:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.369+0000 7f2cc9600700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 0x7f2cac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.370+0000 7f2cc9600700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 0x7f2cac0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005950 tx=0x7f2cb8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.373+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2cbc09dd30 con 0x7f2cc4109fb0 2024-04-20T21:30:02.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:02 smithi016 ceph-mon[21409]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:02.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.610+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2cc402df90 con 0x7f2cc4109fb0 2024-04-20T21:30:02.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.611+0000 7f2cc2000700 1 -- 172.21.15.7:0/1962169049 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f2cbc09d630 con 0x7f2cc4109fb0 2024-04-20T21:30:02.611 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 msgr2=0x7f2cac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 0x7f2cac0a6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8005950 tx=0x7f2cb8009310 comp rx=0 tx=0).stop 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 msgr2=0x7f2cc419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc419fc80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc015000 tx=0x7f2cbc00cfa0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 shutdown_connections 2024-04-20T21:30:02.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cac0a3cc0 0x7f2cac0a6180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cc4109fb0 0x7f2cc419fc80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cc4104ed0 0x7f2cc419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 --2- 172.21.15.7:0/1962169049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc4105880 0x7f2cc419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 >> 172.21.15.7:0/1962169049 conn(0x7f2cc4100760 msgr2=0x7f2cc41012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 shutdown_connections 2024-04-20T21:30:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:02.614+0000 7f2ccc000700 1 -- 172.21.15.7:0/1962169049 wait complete. 2024-04-20T21:30:03.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:03 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1962169049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:03.403 INFO:tasks.cephadm:Adding mds.a on smithi007 2024-04-20T21:30:03.403 INFO:tasks.cephadm:Adding mds.d on smithi007 2024-04-20T21:30:03.403 INFO:tasks.cephadm:Adding mds.g on smithi007 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.j on smithi007 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.b on smithi016 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.e on smithi016 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.h on smithi016 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.k on smithi016 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.c on smithi186 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.f on smithi186 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.i on smithi186 2024-04-20T21:30:03.404 INFO:tasks.cephadm:Adding mds.l on smithi186 2024-04-20T21:30:03.404 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l' 2024-04-20T21:30:03.427 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:03 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1962169049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:03 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1962169049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:03.645 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:30:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:04 smithi016 ceph-mon[21409]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:04.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:04 smithi186 ceph-mon[22157]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:04.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:04 smithi007 ceph-mon[16867]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/230247675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 msgr2=0x7f2cf0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/230247675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0101dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009990 tx=0x7f2ce4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/230247675 shutdown_connections 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/230247675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0101dc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/230247675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0103540 0x7f2cf0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/230247675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf0102ba0 0x7f2cf0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.287 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/230247675 >> 172.21.15.186:0/230247675 conn(0x7f2cf00fcf40 msgr2=0x7f2cf00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/230247675 shutdown_connections 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.286+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/230247675 wait complete. 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 Processor -- start 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 -- start start 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf0102ba0 0x7f2cf01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0103540 0x7f2cf01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010d600 con 0x7f2cf0102ba0 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010d480 con 0x7f2cf0103540 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf010d780 con 0x7f2cf01019a0 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:56410/0 (socket says 172.21.15.186:56410) 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 -- 172.21.15.186:0/4156282467 learned_addr learned my addr 172.21.15.186:0/4156282467 (peer_addr_for_me v2:172.21.15.186:0/0) 2024-04-20T21:30:05.288 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 -- 172.21.15.186:0/4156282467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0103540 msgr2=0x7f2cf01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0103540 0x7f2cf01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 -- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf0102ba0 msgr2=0x7f2cf01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf0102ba0 0x7f2cf01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 -- 172.21.15.186:0/4156282467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf01a09c0 con 0x7f2cf01019a0 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf4b71700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4003d30 con 0x7f2cf01019a0 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce4004830 con 0x7f2cf01019a0 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.287+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf01a0c50 con 0x7f2cf01019a0 2024-04-20T21:30:05.289 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.288+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4005420 con 0x7f2cf01019a0 2024-04-20T21:30:05.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.288+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01a10c0 con 0x7f2cf01019a0 2024-04-20T21:30:05.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.289+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf01a12f0 con 0x7f2cf01019a0 2024-04-20T21:30:05.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.290+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2ce4004cc0 con 0x7f2cf01019a0 2024-04-20T21:30:05.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.290+0000 7f2cedffb700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 0x7f2ce00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.290 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.291+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2ce40d1420 con 0x7f2cf01019a0 2024-04-20T21:30:05.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.291+0000 7f2ceffff700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 0x7f2ce00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.291 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.292+0000 7f2ceffff700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 0x7f2ce00a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097b0 tx=0x7f2cdc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.293 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.294+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ce4013070 con 0x7f2cf01019a0 2024-04-20T21:30:05.499 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.500+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}) -- 0x7f2cf006b7d0 con 0x7f2ce00a3c40 2024-04-20T21:30:05.515 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.517+0000 7f2cedffb700 1 -- 172.21.15.186:0/4156282467 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f2cf006b7d0 con 0x7f2ce00a3c40 2024-04-20T21:30:05.516 INFO:teuthology.orchestra.run.smithi186.stdout:Scheduled mds.all update... 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 msgr2=0x7f2ce00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 0x7f2ce00a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097b0 tx=0x7f2cdc006d20 comp rx=0 tx=0).stop 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 msgr2=0x7f2cf0198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000c00 tx=0x7f2ce40047b0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 shutdown_connections 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2ce00a3c40 0x7f2ce00a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf01019a0 0x7f2cf0198d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.518 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf0103540 0x7f2cf01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 --2- 172.21.15.186:0/4156282467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf0102ba0 0x7f2cf01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 >> 172.21.15.186:0/4156282467 conn(0x7f2cf00fcf40 msgr2=0x7f2cf006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 shutdown_connections 2024-04-20T21:30:05.519 INFO:teuthology.orchestra.run.smithi186.stderr:2024-04-20T21:30:05.519+0000 7f2cf6dd5700 1 -- 172.21.15.186:0/4156282467 wait complete. 2024-04-20T21:30:06.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:06 smithi186 ceph-mon[22157]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:06 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:06.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:06 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:06.252 DEBUG:teuthology.orchestra.run.smithi007:mds.a> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.a.service 2024-04-20T21:30:06.257 DEBUG:teuthology.orchestra.run.smithi007:mds.d> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.d.service 2024-04-20T21:30:06.302 DEBUG:teuthology.orchestra.run.smithi007:mds.g> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.g.service 2024-04-20T21:30:06.311 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:06 smithi007 ceph-mon[16867]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:06.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:06 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:06.312 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:06 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:06.315 DEBUG:teuthology.orchestra.run.smithi007:mds.j> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.j.service 2024-04-20T21:30:06.361 DEBUG:teuthology.orchestra.run.smithi016:mds.b> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.b.service 2024-04-20T21:30:06.364 DEBUG:teuthology.orchestra.run.smithi016:mds.e> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.e.service 2024-04-20T21:30:06.389 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:06 smithi016 ceph-mon[21409]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:06.389 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:06 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:06.389 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:06 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:06.390 DEBUG:teuthology.orchestra.run.smithi016:mds.h> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.h.service 2024-04-20T21:30:06.433 DEBUG:teuthology.orchestra.run.smithi016:mds.k> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.k.service 2024-04-20T21:30:06.476 DEBUG:teuthology.orchestra.run.smithi186:mds.c> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.c.service 2024-04-20T21:30:06.479 DEBUG:teuthology.orchestra.run.smithi186:mds.f> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.f.service 2024-04-20T21:30:06.523 DEBUG:teuthology.orchestra.run.smithi186:mds.i> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.i.service 2024-04-20T21:30:06.565 DEBUG:teuthology.orchestra.run.smithi186:mds.l> sudo journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.l.service 2024-04-20T21:30:06.609 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-20T21:30:06.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.833+0000 7f7036600640 1 -- 172.21.15.7:0/1384677804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 msgr2=0x7f7030102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.833+0000 7f7036600640 1 --2- 172.21.15.7:0/1384677804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f7030102230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7028009ca0 tx=0x7f702802c100 comp rx=0 tx=0).stop 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 -- 172.21.15.7:0/1384677804 shutdown_connections 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 --2- 172.21.15.7:0/1384677804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f7030102230 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 --2- 172.21.15.7:0/1384677804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f7030101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 --2- 172.21.15.7:0/1384677804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f70301002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 -- 172.21.15.7:0/1384677804 >> 172.21.15.7:0/1384677804 conn(0x7f70300fb7a0 msgr2=0x7f70300fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 -- 172.21.15.7:0/1384677804 shutdown_connections 2024-04-20T21:30:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.834+0000 7f7036600640 1 -- 172.21.15.7:0/1384677804 wait complete. 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 Processor -- start 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 -- start start 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f703019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f70301a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f703010fe30 con 0x7f7030100ff0 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f703010fcb0 con 0x7f70300ffec0 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7036600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f703010ffb0 con 0x7f7030101990 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.835+0000 7f7027e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f7027e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57882/0 (socket says 172.21.15.7:57882) 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f7027e00640 1 -- 172.21.15.7:0/2461930447 learned_addr learned my addr 172.21.15.7:0/2461930447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f703019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702fe00640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f70301a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 -- 172.21.15.7:0/2461930447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 msgr2=0x7f70301a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f70301a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 -- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 msgr2=0x7f703019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 -- 172.21.15.7:0/2461930447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70301a6420 con 0x7f70300ffec0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f7027e00640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702f400640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f703019e800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f702000e9f0 tx=0x7f702000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.836+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702000ccc0 con 0x7f70300ffec0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.837+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7020005020 con 0x7f70300ffec0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.837+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70200049e0 con 0x7f70300ffec0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.837+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70301a6710 con 0x7f70300ffec0 2024-04-20T21:30:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.837+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70301a6c60 con 0x7f70300ffec0 2024-04-20T21:30:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.838+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7030107f50 con 0x7f70300ffec0 2024-04-20T21:30:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.840+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f7020005560 con 0x7f70300ffec0 2024-04-20T21:30:06.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.841+0000 7f702cc00640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:06.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.841+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f70200cfee0 con 0x7f70300ffec0 2024-04-20T21:30:06.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.841+0000 7f7027e00640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:06.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.842+0000 7f7027e00640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 0x7f70140a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f703019fd10 tx=0x7f7018005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:06.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:06.846+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f70200051d0 con 0x7f70300ffec0 2024-04-20T21:30:07.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.054+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7030062b60 con 0x7f70300ffec0 2024-04-20T21:30:07.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.056+0000 7f702cc00640 1 -- 172.21.15.7:0/2461930447 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18340 (secure 0 0 0) 0x7f702009ccc0 con 0x7f70300ffec0 2024-04-20T21:30:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:07.056 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:29:55.003374+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 msgr2=0x7f70140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 0x7f70140a60f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f703019fd10 tx=0x7f7018005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 msgr2=0x7f703019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f703019e800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f702000e9f0 tx=0x7f702000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 shutdown_connections 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f70140a3c30 0x7f70140a60f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7030101990 0x7f70301a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7030100ff0 0x7f703019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 --2- 172.21.15.7:0/2461930447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70300ffec0 0x7f703019e800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 >> 172.21.15.7:0/2461930447 conn(0x7f70300fb7a0 msgr2=0x7f70300fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 shutdown_connections 2024-04-20T21:30:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.059+0000 7f7036600640 1 -- 172.21.15.7:0/2461930447 wait complete. 2024-04-20T21:30:07.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:07.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 -- 172.21.15.7:0/1051969354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0514105d60 msgr2=0x7f051410c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 --2- 172.21.15.7:0/1051969354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0514105d60 0x7f051410c5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f050c009ca0 tx=0x7f050c02c100 comp rx=0 tx=0).stop 2024-04-20T21:30:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 -- 172.21.15.7:0/1051969354 shutdown_connections 2024-04-20T21:30:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 --2- 172.21.15.7:0/1051969354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f0514105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 --2- 172.21.15.7:0/1051969354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0514105d60 0x7f051410c5f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 --2- 172.21.15.7:0/1051969354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f05141041b0 0x7f05141045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 -- 172.21.15.7:0/1051969354 >> 172.21.15.7:0/1051969354 conn(0x7f05140ff6e0 msgr2=0x7f0514101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 -- 172.21.15.7:0/1051969354 shutdown_connections 2024-04-20T21:30:07.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.290+0000 7f051c000640 1 -- 172.21.15.7:0/1051969354 wait complete. 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.291+0000 7f051c000640 1 Processor -- start 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.291+0000 7f051c000640 1 -- start start 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 0x7f051419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f051419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 0x7f05141a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f051410fe40 con 0x7f05141041b0 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f051410fcc0 con 0x7f05141053a0 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051c000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f051410ffc0 con 0x7f0514105d60 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f0519600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 0x7f051419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f0518c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f051419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 0x7f05141a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f0519600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 0x7f051419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57892/0 (socket says 172.21.15.7:57892) 2024-04-20T21:30:07.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f0519600640 1 -- 172.21.15.7:0/337713975 learned_addr learned my addr 172.21.15.7:0/337713975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 -- 172.21.15.7:0/337713975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 msgr2=0x7f051419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f051419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 -- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 msgr2=0x7f051419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 0x7f051419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f051a000640 1 -- 172.21.15.7:0/337713975 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05141a2030 con 0x7f0514105d60 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.292+0000 7f0518c00640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f051419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f051a000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 0x7f05141a19c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f050c037040 tx=0x7f050c035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c039040 con 0x7f0514105d60 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c044070 con 0x7f0514105d60 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c004350 con 0x7f0514105d60 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05141a22c0 con 0x7f0514105d60 2024-04-20T21:30:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.293+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05141a27d0 con 0x7f0514105d60 2024-04-20T21:30:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.294+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051410b470 con 0x7f0514105d60 2024-04-20T21:30:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.296+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f050c0044f0 con 0x7f0514105d60 2024-04-20T21:30:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.298+0000 7f0512000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 0x7f04fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.298+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f050c0f2f60 con 0x7f0514105d60 2024-04-20T21:30:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.298+0000 7f0519600640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 0x7f04fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.298+0000 7f0519600640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 0x7f04fc0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f0508005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.300+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f050c0bfe60 con 0x7f0514105d60 2024-04-20T21:30:07.422 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:07.423 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:07 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:07.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:07.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:07 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.497+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f051402d5d0 con 0x7f0514105d60 2024-04-20T21:30:07.497 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:30:07.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.498+0000 7f0512000640 1 -- 172.21.15.7:0/337713975 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f050c0bf760 con 0x7f0514105d60 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 msgr2=0x7f04fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 0x7f04fc0a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f0508005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 msgr2=0x7f05141a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 0x7f05141a19c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f050c037040 tx=0x7f050c035d10 comp rx=0 tx=0).stop 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 shutdown_connections 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04fc0a3c30 0x7f04fc0a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05141053a0 0x7f051419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0514105d60 0x7f05141a19c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 --2- 172.21.15.7:0/337713975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05141041b0 0x7f051419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.500+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 >> 172.21.15.7:0/337713975 conn(0x7f05140ff6e0 msgr2=0x7f0514108640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:07.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.501+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 shutdown_connections 2024-04-20T21:30:07.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.501+0000 7f051c000640 1 -- 172.21.15.7:0/337713975 wait complete. 2024-04-20T21:30:07.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:30:07.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: Saving service mds.all spec with placement smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-20T21:30:07.516 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:07.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:07.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:07.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:07.517 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:07 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:07.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 -- 172.21.15.7:0/4090417389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 msgr2=0x7fa8c8108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/4090417389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c8108920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009900 tx=0x7fa8b802bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 -- 172.21.15.7:0/4090417389 shutdown_connections 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/4090417389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 0x7fa8c81108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/4090417389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/4090417389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c8108920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 -- 172.21.15.7:0/4090417389 >> 172.21.15.7:0/4090417389 conn(0x7fa8c8077450 msgr2=0x7fa8c8079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 -- 172.21.15.7:0/4090417389 shutdown_connections 2024-04-20T21:30:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.781+0000 7fa8c7e00640 1 -- 172.21.15.7:0/4090417389 wait complete. 2024-04-20T21:30:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.782+0000 7fa8c7e00640 1 Processor -- start 2024-04-20T21:30:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 -- start start 2024-04-20T21:30:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 0x7fa8c81a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8114190 con 0x7fa8c81096a0 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8114010 con 0x7fa8c8108500 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8114310 con 0x7fa8c810a060 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45844/0 (socket says 172.21.15.7:45844) 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.783+0000 7fa8c6a00640 1 -- 172.21.15.7:0/177530298 learned_addr learned my addr 172.21.15.7:0/177530298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8bfe00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c7400640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 0x7fa8c81a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 -- 172.21.15.7:0/177530298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 msgr2=0x7fa8c81a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 0x7fa8c81a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 -- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 msgr2=0x7fa8c819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 -- 172.21.15.7:0/177530298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c81a63a0 con 0x7fa8c8108500 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8bfe00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:07.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.784+0000 7fa8c6a00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b802c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.785+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8043020 con 0x7fa8c8108500 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.785+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b80054e0 con 0x7fa8c8108500 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.785+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c81a6630 con 0x7fa8c8108500 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.785+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8003ce0 con 0x7fa8c8108500 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.785+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81a6ba0 con 0x7fa8c8108500 2024-04-20T21:30:07.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.786+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c810f7d0 con 0x7fa8c8108500 2024-04-20T21:30:07.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.788+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fa8b8046020 con 0x7fa8c8108500 2024-04-20T21:30:07.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.789+0000 7fa8bf400640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 0x7fa8ac0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:07.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.790+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8b80f20d0 con 0x7fa8c8108500 2024-04-20T21:30:07.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.790+0000 7fa8bfe00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 0x7fa8ac0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:07.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.790+0000 7fa8bfe00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 0x7fa8ac0a60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00058c0 tx=0x7fa8b0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:07.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:07.794+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa8b804d020 con 0x7fa8c8108500 2024-04-20T21:30:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.045+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c802d7f0 con 0x7fa8c8108500 2024-04-20T21:30:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.046+0000 7fa8bf400640 1 -- 172.21.15.7:0/177530298 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa8b80beeb0 con 0x7fa8c8108500 2024-04-20T21:30:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:08.045 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-20T21:30:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 1 2024-04-20T21:30:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 msgr2=0x7fa8ac0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 0x7fa8ac0a60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa8b00058c0 tx=0x7fa8b0005850 comp rx=0 tx=0).stop 2024-04-20T21:30:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 msgr2=0x7fa8c819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8000c00 tx=0x7fa8b802c6c0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 shutdown_connections 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fa8ac0a3c20 0x7fa8ac0a60e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8c810a060 0x7fa8c81a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c81096a0 0x7fa8c819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 --2- 172.21.15.7:0/177530298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8c8108500 0x7fa8c819e780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.048+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 >> 172.21.15.7:0/177530298 conn(0x7fa8c8077450 msgr2=0x7fa8c810c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.049+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 shutdown_connections 2024-04-20T21:30:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:08.049+0000 7fa8c7e00640 1 -- 172.21.15.7:0/177530298 wait complete. 2024-04-20T21:30:08.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:08 smithi016 ceph-mon[21409]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:08 smithi016 ceph-mon[21409]: Deploying daemon mds.l on smithi186 2024-04-20T21:30:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2461930447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/337713975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/177530298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:08.458 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:08 smithi186 ceph-mon[22157]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:08.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:08 smithi186 ceph-mon[22157]: Deploying daemon mds.l on smithi186 2024-04-20T21:30:08.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2461930447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:08.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/337713975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:08.459 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/177530298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:08.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:08 smithi007 ceph-mon[16867]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:08.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:08 smithi007 ceph-mon[16867]: Deploying daemon mds.l on smithi186 2024-04-20T21:30:08.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2461930447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:08.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/337713975' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:08.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/177530298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:09.216 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 systemd[1]: Starting Ceph mds.l for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:09.494 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 podman[52464]: 2024-04-20 21:30:09.139327676 +0000 UTC m=+0.018581755 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:09.494 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 podman[52464]: 2024-04-20 21:30:09.260247424 +0000 UTC m=+0.139501471 container create caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-20T21:30:09.494 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 podman[52464]: 2024-04-20 21:30:09.496124141 +0000 UTC m=+0.375378191 container init caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD) 2024-04-20T21:30:09.965 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 podman[52464]: 2024-04-20 21:30:09.498723471 +0000 UTC m=+0.377977529 container start caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-20T21:30:09.966 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 ceph-mds[52483]: -- 172.21.15.186:0/4058316181 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55612f083a40 con 0x55612f056c00 2024-04-20T21:30:09.966 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l[52479]: starting mds.l at 2024-04-20T21:30:09.966 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 bash[52464]: caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 2024-04-20T21:30:09.966 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:30:09 smithi186 systemd[1]: Started Ceph mds.l for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:10.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 -- 172.21.15.7:0/2654048479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc105290 msgr2=0x7f69fc105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 --2- 172.21.15.7:0/2654048479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc105290 0x7f69fc105710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009900 tx=0x7f69e402bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 -- 172.21.15.7:0/2654048479 shutdown_connections 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 --2- 172.21.15.7:0/2654048479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc10c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 --2- 172.21.15.7:0/2654048479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc105290 0x7f69fc105710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 --2- 172.21.15.7:0/2654048479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc1040a0 0x7f69fc1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 -- 172.21.15.7:0/2654048479 >> 172.21.15.7:0/2654048479 conn(0x7f69fc0ff630 msgr2=0x7f69fc101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 -- 172.21.15.7:0/2654048479 shutdown_connections 2024-04-20T21:30:10.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.292+0000 7f6a01600640 1 -- 172.21.15.7:0/2654048479 wait complete. 2024-04-20T21:30:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.293+0000 7f6a01600640 1 Processor -- start 2024-04-20T21:30:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 -- start start 2024-04-20T21:30:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 0x7f69fc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10fd80 con 0x7f69fc105c50 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10fc00 con 0x7f69fc105290 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a01600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc10ff00 con 0x7f69fc1040a0 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45866/0 (socket says 172.21.15.7:45866) 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f69f3e00640 1 -- 172.21.15.7:0/1636281719 learned_addr learned my addr 172.21.15.7:0/1636281719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f6a00c00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.294+0000 7f69fbe00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 0x7f69fc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 -- 172.21.15.7:0/1636281719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 msgr2=0x7f69fc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 0x7f69fc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 -- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 msgr2=0x7f69fc1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 -- 172.21.15.7:0/1636281719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc1a1fa0 con 0x7f69fc105290 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f6a00c00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69fbe00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 0x7f69fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f3e00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69e402c8f0 tx=0x7f69e402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4041070 con 0x7f69fc105290 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4004450 con 0x7f69fc105290 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc1a2230 con 0x7f69fc105290 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4034440 con 0x7f69fc105290 2024-04-20T21:30:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.295+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1a2770 con 0x7f69fc105290 2024-04-20T21:30:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.297+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc10b3c0 con 0x7f69fc105290 2024-04-20T21:30:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.298+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f69e4044070 con 0x7f69fc105290 2024-04-20T21:30:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.300+0000 7f69f9600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 0x7f69e00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.300+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f69e40f1bb0 con 0x7f69fc105290 2024-04-20T21:30:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.300+0000 7f69fbe00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 0x7f69e00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.301+0000 7f69fbe00640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 0x7f69e00a60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69fc1050f0 tx=0x7f69ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.304+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f69e40be990 con 0x7f69fc105290 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6832/3154608825,v1:172.21.15.186:6833/3154608825] up:boot 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: fsmap 1 up:standby 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.361 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:10.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:10.362 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:10 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:10.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6832/3154608825,v1:172.21.15.186:6833/3154608825] up:boot 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: fsmap 1 up:standby 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:10.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:10 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6832/3154608825,v1:172.21.15.186:6833/3154608825] up:boot 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: fsmap 1 up:standby 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:10.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:10 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.550+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69fc062b60 con 0x7f69fc105290 2024-04-20T21:30:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.551+0000 7f69f9600640 1 -- 172.21.15.7:0/1636281719 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f69e40be290 con 0x7f69fc105290 2024-04-20T21:30:10.550 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:10.550 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-04-20T21:30:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 msgr2=0x7f69e00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 0x7f69e00a60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69fc1050f0 tx=0x7f69ec005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 msgr2=0x7f69fc19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f69e402c8f0 tx=0x7f69e402c660 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 shutdown_connections 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f69e00a3c20 0x7f69e00a60e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69fc105290 0x7f69fc19a8c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc105c50 0x7f69fc1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 --2- 172.21.15.7:0/1636281719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69fc1040a0 0x7f69fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.553+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 >> 172.21.15.7:0/1636281719 conn(0x7f69fc0ff630 msgr2=0x7f69fc108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.554+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 shutdown_connections 2024-04-20T21:30:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:10.554+0000 7f6a01600640 1 -- 172.21.15.7:0/1636281719 wait complete. 2024-04-20T21:30:11.379 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:11 smithi016 ceph-mon[21409]: Deploying daemon mds.k on smithi016 2024-04-20T21:30:11.380 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:11 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1636281719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:11.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:11 smithi186 ceph-mon[22157]: Deploying daemon mds.k on smithi016 2024-04-20T21:30:11.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:11 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1636281719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:11.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:11 smithi007 ceph-mon[16867]: Deploying daemon mds.k on smithi016 2024-04-20T21:30:11.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:11 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1636281719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:12.082 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:11 smithi016 systemd[1]: Starting Ceph mds.k for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:12.420 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:12 smithi016 ceph-mon[21409]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:12.421 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 podman[50739]: 2024-04-20 21:30:12.117423246 +0000 UTC m=+0.018641965 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:12.421 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 podman[50739]: 2024-04-20 21:30:12.219816447 +0000 UTC m=+0.121035191 container create 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-20T21:30:12.421 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 podman[50739]: 2024-04-20 21:30:12.422289197 +0000 UTC m=+0.323507901 container init 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T21:30:12.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:12 smithi186 ceph-mon[22157]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:12.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:12 smithi007 ceph-mon[16867]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:12.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:12.694 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 podman[50739]: 2024-04-20 21:30:12.426147394 +0000 UTC m=+0.327366098 container start 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:30:12.695 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 ceph-mds[50759]: -- 172.21.15.16:0/1175913427 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555824079a40 con 0x55582404c800 2024-04-20T21:30:12.695 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k[50754]: starting mds.k at 2024-04-20T21:30:12.695 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 bash[50739]: 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 2024-04-20T21:30:12.695 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:30:12 smithi016 systemd[1]: Started Ceph mds.k for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 -- 172.21.15.7:0/652880960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 msgr2=0x7fd03410c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 --2- 172.21.15.7:0/652880960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd03410c670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009ca0 tx=0x7fd02c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 -- 172.21.15.7:0/652880960 shutdown_connections 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 --2- 172.21.15.7:0/652880960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd03410c670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 --2- 172.21.15.7:0/652880960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034105420 0x7fd0341058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 --2- 172.21.15.7:0/652880960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034104230 0x7fd034104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.792+0000 7fd03c600640 1 -- 172.21.15.7:0/652880960 >> 172.21.15.7:0/652880960 conn(0x7fd0340ff7e0 msgr2=0x7fd034101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.793+0000 7fd03c600640 1 -- 172.21.15.7:0/652880960 shutdown_connections 2024-04-20T21:30:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.793+0000 7fd03c600640 1 -- 172.21.15.7:0/652880960 wait complete. 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 Processor -- start 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 -- start start 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 0x7fd03419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd0341a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410fe50 con 0x7fd034104230 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410fcd0 con 0x7fd034105420 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.794+0000 7fd03c600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410ffd0 con 0x7fd034105de0 2024-04-20T21:30:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45882/0 (socket says 172.21.15.7:45882) 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 -- 172.21.15.7:0/768318093 learned_addr learned my addr 172.21.15.7:0/768318093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039c00640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 0x7fd03419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd03a600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd0341a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 -- 172.21.15.7:0/768318093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 msgr2=0x7fd0341a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd0341a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 -- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 msgr2=0x7fd03419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 0x7fd03419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 -- 172.21.15.7:0/768318093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0341a2040 con 0x7fd034105420 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039c00640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 0x7fd03419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.795+0000 7fd039200640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd02000e9f0 tx=0x7fd02000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.796+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02000cd80 con 0x7fd034105420 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.796+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd020010910 con 0x7fd034105420 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.796+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0341a2330 con 0x7fd034105420 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.796+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020018990 con 0x7fd034105420 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.796+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a2880 con 0x7fd034105420 2024-04-20T21:30:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.797+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03410b4f0 con 0x7fd034105420 2024-04-20T21:30:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.800+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fd020010ac0 con 0x7fd034105420 2024-04-20T21:30:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.801+0000 7fd032000640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.802+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0200d0480 con 0x7fd034105420 2024-04-20T21:30:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.802+0000 7fd039c00640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.802+0000 7fd039c00640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd034105280 tx=0x7fd028005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:12.806+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd02009d260 con 0x7fd034105420 2024-04-20T21:30:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.058+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd034062b90 con 0x7fd034105420 2024-04-20T21:30:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.059+0000 7fd032000640 1 -- 172.21.15.7:0/768318093 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd02009cb60 con 0x7fd034105420 2024-04-20T21:30:13.058 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:13.058 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.061+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 msgr2=0x7fd01c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd034105280 tx=0x7fd028005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 msgr2=0x7fd03419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd02000e9f0 tx=0x7fd02000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 shutdown_connections 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd034105420 0x7fd03419a960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd034105de0 0x7fd0341a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 --2- 172.21.15.7:0/768318093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd034104230 0x7fd03419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 >> 172.21.15.7:0/768318093 conn(0x7fd0340ff7e0 msgr2=0x7fd0341086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 shutdown_connections 2024-04-20T21:30:13.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:13.062+0000 7fd03c600640 1 -- 172.21.15.7:0/768318093 wait complete. 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: Deploying daemon mds.j on smithi007 2024-04-20T21:30:13.856 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/768318093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:13.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6832/1030631519,v1:172.21.15.16:6833/1030631519] up:boot 2024-04-20T21:30:13.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: fsmap 2 up:standby 2024-04-20T21:30:13.857 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:13 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: Deploying daemon mds.j on smithi007 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/768318093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:13.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6832/1030631519,v1:172.21.15.16:6833/1030631519] up:boot 2024-04-20T21:30:13.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: fsmap 2 up:standby 2024-04-20T21:30:13.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:13 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: Deploying daemon mds.j on smithi007 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/768318093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6832/1030631519,v1:172.21.15.16:6833/1030631519] up:boot 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: fsmap 2 up:standby 2024-04-20T21:30:13.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:13 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:30:15.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 -- 172.21.15.7:0/2272792401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 msgr2=0x7f233c1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 --2- 172.21.15.7:0/2272792401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1095f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2330009900 tx=0x7f233002bc50 comp rx=0 tx=0).stop 2024-04-20T21:30:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 -- 172.21.15.7:0/2272792401 shutdown_connections 2024-04-20T21:30:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 --2- 172.21.15.7:0/2272792401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 0x7f233c111a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 --2- 172.21.15.7:0/2272792401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 --2- 172.21.15.7:0/2272792401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1095f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 -- 172.21.15.7:0/2272792401 >> 172.21.15.7:0/2272792401 conn(0x7f233c06ef70 msgr2=0x7f233c0713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 -- 172.21.15.7:0/2272792401 shutdown_connections 2024-04-20T21:30:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.410+0000 7f2344600640 1 -- 172.21.15.7:0/2272792401 wait complete. 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.411+0000 7f2344600640 1 Processor -- start 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 -- start start 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c1a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1a3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 0x7f233c1aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f233c114290 con 0x7f233c1091d0 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f233c114110 con 0x7f233c10f620 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2344600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f233c114410 con 0x7f233c073120 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2341200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.412+0000 7f2341200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1a3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33162/0 (socket says 172.21.15.7:33162) 2024-04-20T21:30:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341200640 1 -- 172.21.15.7:0/2789440086 learned_addr learned my addr 172.21.15.7:0/2789440086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c1a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2342600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 0x7f233c1aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 -- 172.21.15.7:0/2789440086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 msgr2=0x7f233c1aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 0x7f233c1aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 -- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 msgr2=0x7f233c1a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 -- 172.21.15.7:0/2789440086 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c1aa940 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.413+0000 7f2341c00640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c1a2d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2330036040 tx=0x7f2330004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.414+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330043020 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.414+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23300054e0 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.414+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c1aabd0 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.414+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330003ce0 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.414+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c1ab0f0 con 0x7f233c073120 2024-04-20T21:30:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.415+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233c068e10 con 0x7f233c073120 2024-04-20T21:30:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.418+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2330046020 con 0x7f233c073120 2024-04-20T21:30:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.419+0000 7f233a000640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 0x7f23240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.419+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23300f2850 con 0x7f233c073120 2024-04-20T21:30:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.419+0000 7f2341200640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 0x7f23240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.420+0000 7f2341200640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 0x7f23240a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f23280095f0 tx=0x7f2328005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.421+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f233004d020 con 0x7f233c073120 2024-04-20T21:30:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.673+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f233c1ab4f0 con 0x7f233c073120 2024-04-20T21:30:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.674+0000 7f233a000640 1 -- 172.21.15.7:0/2789440086 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f23300bf630 con 0x7f233c073120 2024-04-20T21:30:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:15.673 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-20T21:30:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 msgr2=0x7f23240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 0x7f23240a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f23280095f0 tx=0x7f2328005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 msgr2=0x7f233c1a2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c1a2d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2330036040 tx=0x7f2330004e30 comp rx=0 tx=0).stop 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 shutdown_connections 2024-04-20T21:30:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23240a3bb0 0x7f23240a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233c10f620 0x7f233c1aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233c1091d0 0x7f233c1a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 --2- 172.21.15.7:0/2789440086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f233c073120 0x7f233c1a2d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.676+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 >> 172.21.15.7:0/2789440086 conn(0x7f233c06ef70 msgr2=0x7f233c070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.677+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 shutdown_connections 2024-04-20T21:30:15.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:15.677+0000 7f2344600640 1 -- 172.21.15.7:0/2789440086 wait complete. 2024-04-20T21:30:15.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:15 smithi016 ceph-mon[21409]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:15.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:15 smithi186 ceph-mon[22157]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:16.009 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:15 smithi007 ceph-mon[16867]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:16.284 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 podman[54863]: 2024-04-20 21:30:15.92508268 +0000 UTC m=+0.037915777 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:16.595 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 podman[54863]: 2024-04-20 21:30:16.28871778 +0000 UTC m=+0.401550929 container init 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-20T21:30:16.595 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 podman[54863]: 2024-04-20 21:30:16.300230117 +0000 UTC m=+0.413063205 container start 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-20T21:30:16.595 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mds[54881]: -- 172.21.15.7:0/4271882585 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae32009a40 con 0x55ae31fdcc00 2024-04-20T21:30:16.596 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j[54877]: starting mds.j at 2024-04-20T21:30:16.596 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 bash[54863]: 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec 2024-04-20T21:30:16.596 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:30:16 smithi007 systemd[1]: Started Ceph mds.j for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2789440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:16.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:16 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2789440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:16.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:16.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:16.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:16 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2789440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:16 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:17.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:17.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.913+0000 7fae49400640 1 -- 172.21.15.7:0/1078267253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44101e60 msgr2=0x7fae44102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.913+0000 7fae49400640 1 --2- 172.21.15.7:0/1078267253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44101e60 0x7fae44102280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae34009ca0 tx=0x7fae3402c100 comp rx=0 tx=0).stop 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 -- 172.21.15.7:0/1078267253 shutdown_connections 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 --2- 172.21.15.7:0/1078267253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44103040 0x7fae441034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 --2- 172.21.15.7:0/1078267253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 0x7fae4410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 --2- 172.21.15.7:0/1078267253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44101e60 0x7fae44102280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 -- 172.21.15.7:0/1078267253 >> 172.21.15.7:0/1078267253 conn(0x7fae440fd420 msgr2=0x7fae440ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 -- 172.21.15.7:0/1078267253 shutdown_connections 2024-04-20T21:30:17.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.914+0000 7fae49400640 1 -- 172.21.15.7:0/1078267253 wait complete. 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 Processor -- start 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 -- start start 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 0x7fae4419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 0x7fae441a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae4410db40 con 0x7fae44103040 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae4410d9c0 con 0x7fae44101e60 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae49400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fae4410dcc0 con 0x7fae44103a00 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae43400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae43e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 0x7fae4419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:17.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae43400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33198/0 (socket says 172.21.15.7:33198) 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.915+0000 7fae43400640 1 -- 172.21.15.7:0/1201227600 learned_addr learned my addr 172.21.15.7:0/1201227600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae48a00640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 0x7fae441a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 -- 172.21.15.7:0/1201227600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 msgr2=0x7fae441a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 0x7fae441a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 -- 172.21.15.7:0/1201227600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 msgr2=0x7fae4419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 0x7fae4419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 -- 172.21.15.7:0/1201227600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441a2030 con 0x7fae44103040 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43e00640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 0x7fae4419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.916+0000 7fae43400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d950 tx=0x7fae2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.917+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c017070 con 0x7fae44103040 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.917+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae441a2320 con 0x7fae44103040 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.917+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a28c0 con 0x7fae44103040 2024-04-20T21:30:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.917+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2c009c30 con 0x7fae44103040 2024-04-20T21:30:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.917+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c020060 con 0x7fae44103040 2024-04-20T21:30:17.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.918+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae20005290 con 0x7fae44103040 2024-04-20T21:30:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.921+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fae2c010460 con 0x7fae44103040 2024-04-20T21:30:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.922+0000 7fae40c00640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 0x7fae280a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.922+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fae2c0cf780 con 0x7fae44103040 2024-04-20T21:30:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.922+0000 7fae43e00640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 0x7fae280a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:17.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.923+0000 7fae43e00640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 0x7fae280a6060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae3402c640 tx=0x7fae34009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:17.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:17.927+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fae2c009de0 con 0x7fae44103040 2024-04-20T21:30:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:17 smithi016 ceph-mon[21409]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:17 smithi016 ceph-mon[21409]: Deploying daemon mds.d on smithi007 2024-04-20T21:30:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:17 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.7:6834/3500904086,v1:172.21.15.7:6835/3500904086] up:boot 2024-04-20T21:30:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:17 smithi016 ceph-mon[21409]: fsmap 3 up:standby 2024-04-20T21:30:17.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:17 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:17 smithi186 ceph-mon[22157]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:17 smithi186 ceph-mon[22157]: Deploying daemon mds.d on smithi007 2024-04-20T21:30:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:17 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.7:6834/3500904086,v1:172.21.15.7:6835/3500904086] up:boot 2024-04-20T21:30:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:17 smithi186 ceph-mon[22157]: fsmap 3 up:standby 2024-04-20T21:30:17.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:17 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:18.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:17 smithi007 ceph-mon[16867]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:18.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:17 smithi007 ceph-mon[16867]: Deploying daemon mds.d on smithi007 2024-04-20T21:30:18.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:17 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.7:6834/3500904086,v1:172.21.15.7:6835/3500904086] up:boot 2024-04-20T21:30:18.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:17 smithi007 ceph-mon[16867]: fsmap 3 up:standby 2024-04-20T21:30:18.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:17 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:30:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.183+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae20005580 con 0x7fae44103040 2024-04-20T21:30:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.184+0000 7fae40c00640 1 -- 172.21.15.7:0/1201227600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fae2c09bf10 con 0x7fae44103040 2024-04-20T21:30:18.183 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:18.184 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-20T21:30:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 4 2024-04-20T21:30:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 msgr2=0x7fae280a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 0x7fae280a6060 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae3402c640 tx=0x7fae34009360 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 msgr2=0x7fae4419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d950 tx=0x7fae2c00de10 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 shutdown_connections 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fae280a3ba0 0x7fae280a6060 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae44103a00 0x7fae441a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae44103040 0x7fae4419a9a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 --2- 172.21.15.7:0/1201227600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae44101e60 0x7fae4419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.187+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 >> 172.21.15.7:0/1201227600 conn(0x7fae440fd420 msgr2=0x7fae44074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.188+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 shutdown_connections 2024-04-20T21:30:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:18.188+0000 7fae49400640 1 -- 172.21.15.7:0/1201227600 wait complete. 2024-04-20T21:30:18.827 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:18 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1201227600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:18.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:18 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1201227600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:18 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1201227600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:19.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:19 smithi016 ceph-mon[21409]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:19 smithi186 ceph-mon[22157]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:20.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:19 smithi007 ceph-mon[16867]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:20.034 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:19 smithi007 systemd[1]: Starting Ceph mds.d for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:20.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:20.350 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 podman[55207]: 2024-04-20 21:30:19.94284307 +0000 UTC m=+0.045486926 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:20.351 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 podman[55207]: 2024-04-20 21:30:20.046750767 +0000 UTC m=+0.149394631 container create d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:30:20.351 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 podman[55207]: 2024-04-20 21:30:20.351031095 +0000 UTC m=+0.453674977 container init d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 -- 172.21.15.7:0/2666811626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0105250 msgr2=0x7fddf0107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 --2- 172.21.15.7:0/2666811626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0105250 0x7fddf0107640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009900 tx=0x7fdddc02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 -- 172.21.15.7:0/2666811626 shutdown_connections 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 --2- 172.21.15.7:0/2666811626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0107b80 0x7fddf0109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 --2- 172.21.15.7:0/2666811626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0105250 0x7fddf0107640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 --2- 172.21.15.7:0/2666811626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0102920 0x7fddf0104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 -- 172.21.15.7:0/2666811626 >> 172.21.15.7:0/2666811626 conn(0x7fddf00fc330 msgr2=0x7fddf00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 -- 172.21.15.7:0/2666811626 shutdown_connections 2024-04-20T21:30:20.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.417+0000 7fddf5000640 1 -- 172.21.15.7:0/2666811626 wait complete. 2024-04-20T21:30:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.418+0000 7fddf5000640 1 Processor -- start 2024-04-20T21:30:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.418+0000 7fddf5000640 1 -- start start 2024-04-20T21:30:20.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 0x7fddf019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0107b80 0x7fddf01a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010db60 con 0x7fddf0102920 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010d9e0 con 0x7fddf0105250 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddf5000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fddf010dce0 con 0x7fddf0107b80 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 0x7fddf019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33216/0 (socket says 172.21.15.7:33216) 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddef400640 1 -- 172.21.15.7:0/732586632 learned_addr learned my addr 172.21.15.7:0/732586632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddefe00640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0107b80 0x7fddf01a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 -- 172.21.15.7:0/732586632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0107b80 msgr2=0x7fddf01a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0107b80 0x7fddf01a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 -- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 msgr2=0x7fddf019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.419+0000 7fddeea00640 1 -- 172.21.15.7:0/732586632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf01a1ff0 con 0x7fddf0105250 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.420+0000 7fddef400640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.420+0000 7fddeea00640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 0x7fddf019a910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009230 tx=0x7fdddc02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.420+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc043020 con 0x7fddf0105250 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.421+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdddc0054e0 con 0x7fddf0105250 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.421+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc003b60 con 0x7fddf0105250 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.421+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf01a2280 con 0x7fddf0105250 2024-04-20T21:30:20.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.421+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf01a2750 con 0x7fddf0105250 2024-04-20T21:30:20.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.422+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf004fd60 con 0x7fddf0105250 2024-04-20T21:30:20.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.425+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fdddc046020 con 0x7fddf0105250 2024-04-20T21:30:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.426+0000 7fdde7e00640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 0x7fddd40a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.427+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdddc0f25d0 con 0x7fddf0105250 2024-04-20T21:30:20.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.427+0000 7fddef400640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 0x7fddd40a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.427+0000 7fddef400640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 0x7fddd40a60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdde0011550 tx=0x7fdde0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.429+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdddc04d020 con 0x7fddf0105250 2024-04-20T21:30:20.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.699+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf01096d0 con 0x7fddf0105250 2024-04-20T21:30:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.700+0000 7fdde7e00640 1 -- 172.21.15.7:0/732586632 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7fddf01096d0 con 0x7fddf0105250 2024-04-20T21:30:20.699 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:20.699 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-20T21:30:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 msgr2=0x7fddd40a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 0x7fddd40a60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdde0011550 tx=0x7fdde0005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 msgr2=0x7fddf019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 0x7fddf019a910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdddc009230 tx=0x7fdddc02c5d0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 shutdown_connections 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fddd40a3c20 0x7fddd40a60e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddf0105250 0x7fddf019a910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fddf0107b80 0x7fddf01a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 --2- 172.21.15.7:0/732586632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddf0102920 0x7fddf019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 >> 172.21.15.7:0/732586632 conn(0x7fddf00fc330 msgr2=0x7fddf0102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 shutdown_connections 2024-04-20T21:30:20.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:20.703+0000 7fddf5000640 1 -- 172.21.15.7:0/732586632 wait complete. 2024-04-20T21:30:20.719 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 podman[55207]: 2024-04-20 21:30:20.364499655 +0000 UTC m=+0.467143538 container start d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:30:20.719 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 ceph-mds[55246]: -- 172.21.15.7:0/3646366537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556900d49a40 con 0x556900d1c800 2024-04-20T21:30:20.719 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d[55222]: starting mds.d at 2024-04-20T21:30:20.720 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 bash[55207]: d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 2024-04-20T21:30:20.720 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:30:20 smithi007 systemd[1]: Started Ceph mds.d for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: Deploying daemon mds.e on smithi016 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.7:6836/2021614032,v1:172.21.15.7:6837/2021614032] up:boot 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: fsmap 4 up:standby 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:21.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:21 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/732586632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: Deploying daemon mds.e on smithi016 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.7:6836/2021614032,v1:172.21.15.7:6837/2021614032] up:boot 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: fsmap 4 up:standby 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:21.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:21 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/732586632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:22.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: Deploying daemon mds.e on smithi016 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.7:6836/2021614032,v1:172.21.15.7:6837/2021614032] up:boot 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: fsmap 4 up:standby 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:30:22.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:21 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/732586632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:22.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 -- 172.21.15.7:0/2891020421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 msgr2=0x7f573c074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 --2- 172.21.15.7:0/2891020421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c074e40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5728009900 tx=0x7f572802bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 -- 172.21.15.7:0/2891020421 shutdown_connections 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 --2- 172.21.15.7:0/2891020421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c109d70 0x7f573c10c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 --2- 172.21.15.7:0/2891020421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c074e40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 --2- 172.21.15.7:0/2891020421 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c075fa0 0x7f573c0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 -- 172.21.15.7:0/2891020421 >> 172.21.15.7:0/2891020421 conn(0x7f573c0fd420 msgr2=0x7f573c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 -- 172.21.15.7:0/2891020421 shutdown_connections 2024-04-20T21:30:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.921+0000 7f5740c00640 1 -- 172.21.15.7:0/2891020421 wait complete. 2024-04-20T21:30:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.922+0000 7f5740c00640 1 Processor -- start 2024-04-20T21:30:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.922+0000 7f5740c00640 1 -- start start 2024-04-20T21:30:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 0x7f573c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fd30 con 0x7f573c0749c0 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10fbb0 con 0x7f573c075fa0 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f5740c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f573c10feb0 con 0x7f573c109d70 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f573aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f573b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f573aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39870/0 (socket says 172.21.15.7:39870) 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f573aa00640 1 -- 172.21.15.7:0/2894906256 learned_addr learned my addr 172.21.15.7:0/2894906256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.923+0000 7f573be00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 0x7f573c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 -- 172.21.15.7:0/2894906256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 msgr2=0x7f573c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 0x7f573c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 -- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 msgr2=0x7f573c19a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c19a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 -- 172.21.15.7:0/2894906256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c1a1fd0 con 0x7f573c075fa0 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573be00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 0x7f573c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573b400640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f573aa00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5728009230 tx=0x7f572802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728041070 con 0x7f573c075fa0 2024-04-20T21:30:22.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5728004450 con 0x7f573c075fa0 2024-04-20T21:30:22.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.924+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5728003db0 con 0x7f573c075fa0 2024-04-20T21:30:22.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.925+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c1a2260 con 0x7f573c075fa0 2024-04-20T21:30:22.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.925+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1a2700 con 0x7f573c075fa0 2024-04-20T21:30:22.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.926+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573c108aa0 con 0x7f573c075fa0 2024-04-20T21:30:22.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.929+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5728044070 con 0x7f573c075fa0 2024-04-20T21:30:22.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.930+0000 7f5733e00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 0x7f57200a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.931+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f57280f1b30 con 0x7f573c075fa0 2024-04-20T21:30:22.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.931+0000 7f573b400640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 0x7f57200a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.931+0000 7f573b400640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 0x7f57200a60e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fc0 tx=0x7f572c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:22.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:22.935+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f57280be910 con 0x7f573c075fa0 2024-04-20T21:30:22.944 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:22 smithi016 systemd[1]: Starting Ceph mds.e for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.178+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f573c02d5d0 con 0x7f573c075fa0 2024-04-20T21:30:23.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.179+0000 7f5733e00640 1 -- 172.21.15.7:0/2894906256 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3865 (secure 0 0 0) 0x7f57280be210 con 0x7f573c075fa0 2024-04-20T21:30:23.177 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:23.178 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-20T21:30:23.178 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 5 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 msgr2=0x7f57200a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 0x7f57200a60e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f572c009fc0 tx=0x7f572c005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 msgr2=0x7f573c19a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5728009230 tx=0x7f572802c650 comp rx=0 tx=0).stop 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 shutdown_connections 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f57200a3c20 0x7f57200a60e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c075fa0 0x7f573c19a8f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f573c109d70 0x7f573c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 --2- 172.21.15.7:0/2894906256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f573c0749c0 0x7f573c19a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 >> 172.21.15.7:0/2894906256 conn(0x7f573c0fd420 msgr2=0x7f573c105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.181+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 shutdown_connections 2024-04-20T21:30:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:23.182+0000 7f5740c00640 1 -- 172.21.15.7:0/2894906256 wait complete. 2024-04-20T21:30:23.359 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 podman[51058]: 2024-04-20 21:30:22.924369166 +0000 UTC m=+0.018536696 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:23.359 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 podman[51058]: 2024-04-20 21:30:23.116873124 +0000 UTC m=+0.211040635 container create f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2894906256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 podman[51058]: 2024-04-20 21:30:23.36100779 +0000 UTC m=+0.455175301 container init f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 podman[51058]: 2024-04-20 21:30:23.36381492 +0000 UTC m=+0.457982438 container start f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-mds[51076]: -- 172.21.15.16:0/2866544925 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a16459a40 con 0x555a1642cc00 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: starting mds.e at 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 bash[51058]: f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 2024-04-20T21:30:23.696 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:30:23 smithi016 systemd[1]: Started Ceph mds.e for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:23.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:23.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2894906256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:23.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:23.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:23.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:23 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:23.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:23.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2894906256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:23.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:23 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:24.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:24 smithi016 ceph-mon[21409]: Deploying daemon mds.f on smithi186 2024-04-20T21:30:24.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:24 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6834/2330254135,v1:172.21.15.16:6835/2330254135] up:boot 2024-04-20T21:30:24.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:24 smithi016 ceph-mon[21409]: fsmap 5 up:standby 2024-04-20T21:30:24.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:24 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:30:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:24 smithi186 ceph-mon[22157]: Deploying daemon mds.f on smithi186 2024-04-20T21:30:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:24 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6834/2330254135,v1:172.21.15.16:6835/2330254135] up:boot 2024-04-20T21:30:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:24 smithi186 ceph-mon[22157]: fsmap 5 up:standby 2024-04-20T21:30:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:24 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:30:25.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:24 smithi007 ceph-mon[16867]: Deploying daemon mds.f on smithi186 2024-04-20T21:30:25.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:24 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6834/2330254135,v1:172.21.15.16:6835/2330254135] up:boot 2024-04-20T21:30:25.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:24 smithi007 ceph-mon[16867]: fsmap 5 up:standby 2024-04-20T21:30:25.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:24 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:30:25.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.423+0000 7f2a01c00640 1 -- 172.21.15.7:0/4195447135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc107b80 msgr2=0x7f29fc109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.423+0000 7f2a01c00640 1 --2- 172.21.15.7:0/4195447135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc107b80 0x7f29fc109fd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009860 tx=0x7f29f402bb70 comp rx=0 tx=0).stop 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 -- 172.21.15.7:0/4195447135 shutdown_connections 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 --2- 172.21.15.7:0/4195447135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc107b80 0x7f29fc109fd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 --2- 172.21.15.7:0/4195447135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc105250 0x7f29fc107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 --2- 172.21.15.7:0/4195447135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 -- 172.21.15.7:0/4195447135 >> 172.21.15.7:0/4195447135 conn(0x7f29fc0fc330 msgr2=0x7f29fc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 -- 172.21.15.7:0/4195447135 shutdown_connections 2024-04-20T21:30:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.424+0000 7f2a01c00640 1 -- 172.21.15.7:0/4195447135 wait complete. 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 Processor -- start 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 -- start start 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc105250 0x7f29fc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc107b80 0x7f29fc1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10db40 con 0x7f29fc102920 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10d9c0 con 0x7f29fc105250 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.425+0000 7f2a01c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc10dcc0 con 0x7f29fc107b80 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42238/0 (socket says 172.21.15.7:42238) 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 -- 172.21.15.7:0/223808417 learned_addr learned my addr 172.21.15.7:0/223808417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f2a01200640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc107b80 0x7f29fc1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fb400640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc105250 0x7f29fc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 -- 172.21.15.7:0/223808417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc107b80 msgr2=0x7f29fc1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc107b80 0x7f29fc1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 -- 172.21.15.7:0/223808417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc105250 msgr2=0x7f29fc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc105250 0x7f29fc19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 -- 172.21.15.7:0/223808417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc1a1fd0 con 0x7f29fc102920 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.426+0000 7f29fbe00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c420 tx=0x7f29ec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.427+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec003970 con 0x7f29fc102920 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.427+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1a22c0 con 0x7f29fc102920 2024-04-20T21:30:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.427+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ec004020 con 0x7f29fc102920 2024-04-20T21:30:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.427+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00bad0 con 0x7f29fc102920 2024-04-20T21:30:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.427+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a2860 con 0x7f29fc102920 2024-04-20T21:30:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.428+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc109670 con 0x7f29fc102920 2024-04-20T21:30:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.430+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f29ec01d020 con 0x7f29fc102920 2024-04-20T21:30:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.432+0000 7f29f8c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 0x7f29e00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.432+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f29ec0cfe60 con 0x7f29fc102920 2024-04-20T21:30:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.432+0000 7f29fb400640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 0x7f29e00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.433+0000 7f29fb400640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 0x7f29e00a6060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29fc19b8c0 tx=0x7f29e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.436+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f29ec09cc40 con 0x7f29fc102920 2024-04-20T21:30:25.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.680+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29fc02d5d0 con 0x7f29fc102920 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.681+0000 7f29f8c00640 1 -- 172.21.15.7:0/223808417 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f29ec09c540 con 0x7f29fc102920 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 6 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 msgr2=0x7f29e00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 0x7f29e00a6060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f29fc19b8c0 tx=0x7f29e4005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 msgr2=0x7f29fc19a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00c420 tx=0x7f29ec009f90 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 shutdown_connections 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f29e00a3ba0 0x7f29e00a6060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29fc105250 0x7f29fc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29fc107b80 0x7f29fc1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 --2- 172.21.15.7:0/223808417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29fc102920 0x7f29fc19a3b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 >> 172.21.15.7:0/223808417 conn(0x7f29fc0fc330 msgr2=0x7f29fc101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 shutdown_connections 2024-04-20T21:30:25.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:25.684+0000 7f2a01c00640 1 -- 172.21.15.7:0/223808417 wait complete. 2024-04-20T21:30:25.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:25 smithi016 ceph-mon[21409]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:25.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:25 smithi186 ceph-mon[22157]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:25.965 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:25 smithi186 systemd[1]: Starting Ceph mds.f for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:26.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:25 smithi007 ceph-mon[16867]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:26.290 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 podman[52769]: 2024-04-20 21:30:25.93496187 +0000 UTC m=+0.018199512 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:26.291 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 podman[52769]: 2024-04-20 21:30:26.048317465 +0000 UTC m=+0.131555091 container create fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-20T21:30:26.590 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 podman[52769]: 2024-04-20 21:30:26.292668674 +0000 UTC m=+0.375906299 container init fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 podman[52769]: 2024-04-20 21:30:26.295309167 +0000 UTC m=+0.378546792 container start fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mds[52788]: -- 172.21.15.186:0/2682474945 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c58b5a5a40 con 0x55c58b578800 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: starting mds.f at 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 bash[52769]: fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:30:26 smithi186 systemd[1]: Started Ceph mds.f for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/223808417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.591 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:26.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:26.592 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:26 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/223808417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:26.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:26 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/223808417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:27.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:26 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:27.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:27 smithi186 ceph-mon[22157]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:27 smithi186 ceph-mon[22157]: Deploying daemon mds.c on smithi186 2024-04-20T21:30:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:27 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6834/2162825015,v1:172.21.15.186:6835/2162825015] up:boot 2024-04-20T21:30:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:27 smithi186 ceph-mon[22157]: fsmap 6 up:standby 2024-04-20T21:30:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:27 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:30:27.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 -- 172.21.15.7:0/378626791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101a10 msgr2=0x7f9b3c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:27.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 --2- 172.21.15.7:0/378626791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101a10 0x7f9b3c101e90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009ca0 tx=0x7f9b3402c100 comp rx=0 tx=0).stop 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 -- 172.21.15.7:0/378626791 shutdown_connections 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 --2- 172.21.15.7:0/378626791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101a10 0x7f9b3c101e90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 --2- 172.21.15.7:0/378626791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101070 0x7f9b3c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.925+0000 7f9b3b200640 1 --2- 172.21.15.7:0/378626791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 0x7f9b3c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.926+0000 7f9b3b200640 1 -- 172.21.15.7:0/378626791 >> 172.21.15.7:0/378626791 conn(0x7f9b3c0fc330 msgr2=0x7f9b3c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.926+0000 7f9b3b200640 1 -- 172.21.15.7:0/378626791 shutdown_connections 2024-04-20T21:30:27.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.926+0000 7f9b3b200640 1 -- 172.21.15.7:0/378626791 wait complete. 2024-04-20T21:30:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 Processor -- start 2024-04-20T21:30:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 -- start start 2024-04-20T21:30:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 0x7f9b3c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101a10 0x7f9b3c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c10fd40 con 0x7f9b3c101070 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c10fbc0 con 0x7f9b3c100710 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b3b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c10fec0 con 0x7f9b3c101a10 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b31400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b31400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42254/0 (socket says 172.21.15.7:42254) 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.927+0000 7f9b31400640 1 -- 172.21.15.7:0/1775436394 learned_addr learned my addr 172.21.15.7:0/1775436394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b39e00640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 0x7f9b3c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b3a800640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101a10 0x7f9b3c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 -- 172.21.15.7:0/1775436394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101a10 msgr2=0x7f9b3c1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101a10 0x7f9b3c1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 -- 172.21.15.7:0/1775436394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 msgr2=0x7f9b3c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 0x7f9b3c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 -- 172.21.15.7:0/1775436394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c1a1fa0 con 0x7f9b3c101070 2024-04-20T21:30:27.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b31400640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cd80 con 0x7f9b3c101070 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.928+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b24010910 con 0x7f9b3c101070 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.929+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c1a2290 con 0x7f9b3c101070 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.929+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24018990 con 0x7f9b3c101070 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.929+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c1a27e0 con 0x7f9b3c101070 2024-04-20T21:30:27.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.930+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3c10b460 con 0x7f9b3c101070 2024-04-20T21:30:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.932+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f9b24010ac0 con 0x7f9b3c101070 2024-04-20T21:30:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.934+0000 7f9b33400640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 0x7f9b200a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.934+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9b240d0320 con 0x7f9b3c101070 2024-04-20T21:30:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.934+0000 7f9b39e00640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 0x7f9b200a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:27.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.935+0000 7f9b39e00640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 0x7f9b200a6060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009bd0 tx=0x7f9b2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:27.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:27.938+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9b2409cab0 con 0x7f9b3c101070 2024-04-20T21:30:27.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:27 smithi016 ceph-mon[21409]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:27.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:27 smithi016 ceph-mon[21409]: Deploying daemon mds.c on smithi186 2024-04-20T21:30:27.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:27 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6834/2162825015,v1:172.21.15.186:6835/2162825015] up:boot 2024-04-20T21:30:27.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:27 smithi016 ceph-mon[21409]: fsmap 6 up:standby 2024-04-20T21:30:27.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:27 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:30:28.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:27 smithi007 ceph-mon[16867]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:28.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:27 smithi007 ceph-mon[16867]: Deploying daemon mds.c on smithi186 2024-04-20T21:30:28.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:27 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6834/2162825015,v1:172.21.15.186:6835/2162825015] up:boot 2024-04-20T21:30:28.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:27 smithi007 ceph-mon[16867]: fsmap 6 up:standby 2024-04-20T21:30:28.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:27 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:30:28.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.186+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b3c02d5d0 con 0x7f9b3c101070 2024-04-20T21:30:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.187+0000 7f9b33400640 1 -- 172.21.15.7:0/1775436394 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f9b240a0db0 con 0x7f9b3c101070 2024-04-20T21:30:28.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:28.186 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-20T21:30:28.186 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.189+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 msgr2=0x7f9b200a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.189+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 0x7f9b200a6060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009bd0 tx=0x7f9b2c005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.189+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 msgr2=0x7f9b3c19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.189+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 shutdown_connections 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9b200a3ba0 0x7f9b200a6060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b3c101a10 0x7f9b3c1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b3c101070 0x7f9b3c19a8c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 --2- 172.21.15.7:0/1775436394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b3c100710 0x7f9b3c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:28.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 >> 172.21.15.7:0/1775436394 conn(0x7f9b3c0fc330 msgr2=0x7f9b3c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 shutdown_connections 2024-04-20T21:30:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:28.190+0000 7f9b3b200640 1 -- 172.21.15.7:0/1775436394 wait complete. 2024-04-20T21:30:28.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:28 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1775436394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:28.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:28 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1775436394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:28.966 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:28 smithi186 systemd[1]: Starting Ceph mds.c for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:29.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:28 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1775436394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:29.305 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 podman[53080]: 2024-04-20 21:30:28.920617791 +0000 UTC m=+0.017886756 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:29.305 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 podman[53080]: 2024-04-20 21:30:29.028789414 +0000 UTC m=+0.126058356 container create 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T21:30:29.305 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 podman[53080]: 2024-04-20 21:30:29.306739779 +0000 UTC m=+0.404008720 container init 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T21:30:29.607 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 podman[53080]: 2024-04-20 21:30:29.309665662 +0000 UTC m=+0.406934614 container start 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-20T21:30:29.608 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mds[53099]: -- 172.21.15.186:0/3420549161 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55794ba7da40 con 0x55794ba50800 2024-04-20T21:30:29.608 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: starting mds.c at 2024-04-20T21:30:29.608 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 bash[53080]: 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 2024-04-20T21:30:29.608 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:30:29 smithi186 systemd[1]: Started Ceph mds.c for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:29.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:29.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:29.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:29.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:29 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:29.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:29.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:29.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:29.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:29.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:29 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:30.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:29 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:30.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 -- 172.21.15.7:0/541857311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24103210 msgr2=0x7f0e24103690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 --2- 172.21.15.7:0/541857311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24103210 0x7f0e24103690 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009900 tx=0x7f0e0c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 -- 172.21.15.7:0/541857311 shutdown_connections 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 --2- 172.21.15.7:0/541857311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103bd0 0x7f0e2410a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 --2- 172.21.15.7:0/541857311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24103210 0x7f0e24103690 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 --2- 172.21.15.7:0/541857311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24102020 0x7f0e24102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 -- 172.21.15.7:0/541857311 >> 172.21.15.7:0/541857311 conn(0x7f0e240fd590 msgr2=0x7f0e240ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 -- 172.21.15.7:0/541857311 shutdown_connections 2024-04-20T21:30:30.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.441+0000 7f0e2b600640 1 -- 172.21.15.7:0/541857311 wait complete. 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.442+0000 7f0e2b600640 1 Processor -- start 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.442+0000 7f0e2b600640 1 -- start start 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.442+0000 7f0e2b600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24102020 0x7f0e2419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.442+0000 7f0e2b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e2b600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24103bd0 0x7f0e241a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e2b600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410dc90 con 0x7f0e24102020 2024-04-20T21:30:30.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e2b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410db10 con 0x7f0e24103210 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e2b600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2410de10 con 0x7f0e24103bd0 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39914/0 (socket says 172.21.15.7:39914) 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e28c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24102020 0x7f0e2419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 -- 172.21.15.7:0/3330057116 learned_addr learned my addr 172.21.15.7:0/3330057116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e29600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24103bd0 0x7f0e241a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 -- 172.21.15.7:0/3330057116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24103bd0 msgr2=0x7f0e241a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24103bd0 0x7f0e241a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 -- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24102020 msgr2=0x7f0e2419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24102020 0x7f0e2419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.443+0000 7f0e23e00640 1 -- 172.21.15.7:0/3330057116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241a20c0 con 0x7f0e24103210 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e23e00640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c043020 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241a2350 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241a28f0 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e0c0054e0 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.444+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c003b40 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.445+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e000052c0 con 0x7f0e24103210 2024-04-20T21:30:30.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.447+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0e0c046020 con 0x7f0e24103210 2024-04-20T21:30:30.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.449+0000 7f0e21600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 0x7f0e080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.449+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0e0c0f18d0 con 0x7f0e24103210 2024-04-20T21:30:30.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.449+0000 7f0e28c00640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 0x7f0e080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.450+0000 7f0e28c00640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 0x7f0e080a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e24103070 tx=0x7f0e14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.453+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0e0c04d030 con 0x7f0e24103210 2024-04-20T21:30:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.706+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e000055b0 con 0x7f0e24103210 2024-04-20T21:30:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.707+0000 7f0e21600640 1 -- 172.21.15.7:0/3330057116 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f0e0c0be6b0 con 0x7f0e24103210 2024-04-20T21:30:30.706 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:30.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T21:30:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 msgr2=0x7f0e080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 0x7f0e080a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e24103070 tx=0x7f0e14005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 msgr2=0x7f0e2419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c000c00 tx=0x7f0e0c02c650 comp rx=0 tx=0).stop 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 shutdown_connections 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0e080a3bb0 0x7f0e080a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e24103210 0x7f0e2419a9e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0e24103bd0 0x7f0e241a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 --2- 172.21.15.7:0/3330057116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e24102020 0x7f0e2419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 >> 172.21.15.7:0/3330057116 conn(0x7f0e240fd590 msgr2=0x7f0e24074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 shutdown_connections 2024-04-20T21:30:30.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:30.710+0000 7f0e2b600640 1 -- 172.21.15.7:0/3330057116 wait complete. 2024-04-20T21:30:30.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:30 smithi016 ceph-mon[21409]: Deploying daemon mds.h on smithi016 2024-04-20T21:30:30.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:30 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:boot 2024-04-20T21:30:30.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:30 smithi016 ceph-mon[21409]: fsmap 7 up:standby 2024-04-20T21:30:30.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:30 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:30:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:30 smithi186 ceph-mon[22157]: Deploying daemon mds.h on smithi016 2024-04-20T21:30:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:30 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:boot 2024-04-20T21:30:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:30 smithi186 ceph-mon[22157]: fsmap 7 up:standby 2024-04-20T21:30:30.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:30 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:30:31.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:30 smithi007 ceph-mon[16867]: Deploying daemon mds.h on smithi016 2024-04-20T21:30:31.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:30 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:boot 2024-04-20T21:30:31.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:30 smithi007 ceph-mon[16867]: fsmap 7 up:standby 2024-04-20T21:30:31.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:30 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:30:31.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:31 smithi016 ceph-mon[21409]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:31.907 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:31 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3330057116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:31.907 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:31 smithi016 systemd[1]: Starting Ceph mds.h for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:31.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:31 smithi186 ceph-mon[22157]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:31.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:31 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3330057116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:32.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:31 smithi007 ceph-mon[16867]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:32.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:31 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3330057116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:32.166 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:31 smithi016 podman[51366]: 2024-04-20 21:30:31.811039887 +0000 UTC m=+0.018406800 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:32.166 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 podman[51366]: 2024-04-20 21:30:32.039424534 +0000 UTC m=+0.246791446 container create ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 podman[51366]: 2024-04-20 21:30:32.291243069 +0000 UTC m=+0.498610005 container init ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 podman[51366]: 2024-04-20 21:30:32.294040785 +0000 UTC m=+0.501407700 container start ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 ceph-mds[51385]: -- 172.21.15.16:0/2800600728 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561764d3ba40 con 0x561764d0ec00 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: starting mds.h at 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 bash[51366]: ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd 2024-04-20T21:30:32.695 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:30:32 smithi016 systemd[1]: Started Ceph mds.h for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:32.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:32.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 -- 172.21.15.7:0/732703919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105290 msgr2=0x7f8d54105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 --2- 172.21.15.7:0/732703919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105290 0x7f8d54105710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009900 tx=0x7f8d3c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 -- 172.21.15.7:0/732703919 shutdown_connections 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 --2- 172.21.15.7:0/732703919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105c50 0x7f8d5410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 --2- 172.21.15.7:0/732703919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105290 0x7f8d54105710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 --2- 172.21.15.7:0/732703919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d541044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 -- 172.21.15.7:0/732703919 >> 172.21.15.7:0/732703919 conn(0x7f8d540ff630 msgr2=0x7f8d54101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 -- 172.21.15.7:0/732703919 shutdown_connections 2024-04-20T21:30:32.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.956+0000 7f8d5a600640 1 -- 172.21.15.7:0/732703919 wait complete. 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.957+0000 7f8d5a600640 1 Processor -- start 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.957+0000 7f8d5a600640 1 -- start start 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 0x7f8d5419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410fd80 con 0x7f8d541040a0 2024-04-20T21:30:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410fc00 con 0x7f8d54105290 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d5a600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5410ff00 con 0x7f8d54105c50 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44508/0 (socket says 172.21.15.7:44508) 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44616/0 (socket says 172.21.15.7:44616) 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59200640 1 -- 172.21.15.7:0/2600783124 learned_addr learned my addr 172.21.15.7:0/2600783124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d53e00640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 0x7f8d5419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.958+0000 7f8d59c00640 1 -- 172.21.15.7:0/2600783124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 msgr2=0x7f8d5419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59c00640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 0x7f8d5419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59c00640 1 -- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 msgr2=0x7f8d5419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59c00640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59c00640 1 -- 172.21.15.7:0/2600783124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d541a1f70 con 0x7f8d54105c50 2024-04-20T21:30:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59200640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d53e00640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 0x7f8d5419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d59c00640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009fd0 tx=0x7f8d4c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00ce20 con 0x7f8d54105c50 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d4c005020 con 0x7f8d54105c50 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c010720 con 0x7f8d54105c50 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d541a2260 con 0x7f8d54105c50 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.959+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d541a2730 con 0x7f8d54105c50 2024-04-20T21:30:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.960+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d5410b360 con 0x7f8d54105c50 2024-04-20T21:30:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.962+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8d4c005540 con 0x7f8d54105c50 2024-04-20T21:30:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.964+0000 7f8d51600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 0x7f8d380a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.964+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8d4c0d0660 con 0x7f8d54105c50 2024-04-20T21:30:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.964+0000 7f8d59200640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 0x7f8d380a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.965+0000 7f8d59200640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 0x7f8d380a6060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005fd0 tx=0x7f8d4400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:32.968+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8d4c09d440 con 0x7f8d54105c50 2024-04-20T21:30:33.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.217+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d5402d5d0 con 0x7f8d54105c50 2024-04-20T21:30:33.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.218+0000 7f8d51600640 1 -- 172.21.15.7:0/2600783124 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f8d4c09cd40 con 0x7f8d54105c50 2024-04-20T21:30:33.217 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:33.217 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-20T21:30:33.217 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-20T21:30:33.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.220+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 msgr2=0x7f8d380a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.220+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 0x7f8d380a6060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d44005fd0 tx=0x7f8d4400b4a0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.220+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 msgr2=0x7f8d541a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.220+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c009fd0 tx=0x7f8d4c00ef10 comp rx=0 tx=0).stop 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 shutdown_connections 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8d380a3ba0 0x7f8d380a6060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d54105290 0x7f8d5419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d54105c50 0x7f8d541a1900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 --2- 172.21.15.7:0/2600783124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d541040a0 0x7f8d5419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 >> 172.21.15.7:0/2600783124 conn(0x7f8d540ff630 msgr2=0x7f8d54108530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 shutdown_connections 2024-04-20T21:30:33.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:33.221+0000 7f8d5a600640 1 -- 172.21.15.7:0/2600783124 wait complete. 2024-04-20T21:30:33.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: Deploying daemon mds.i on smithi186 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6836/4051126294,v1:172.21.15.16:6837/4051126294] up:boot 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: fsmap 8 up:standby 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:30:33.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2600783124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: Deploying daemon mds.i on smithi186 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6836/4051126294,v1:172.21.15.16:6837/4051126294] up:boot 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: fsmap 8 up:standby 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:30:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2600783124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: Deploying daemon mds.i on smithi186 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6836/4051126294,v1:172.21.15.16:6837/4051126294] up:boot 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: fsmap 8 up:standby 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:30:33.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2600783124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:34.959 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:34 smithi186 systemd[1]: Starting Ceph mds.i for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:35.211 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:34 smithi186 podman[53400]: 2024-04-20 21:30:34.862897474 +0000 UTC m=+0.018260359 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:35.211 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:34 smithi186 podman[53400]: 2024-04-20 21:30:34.976777082 +0000 UTC m=+0.132139956 container create 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T21:30:35.211 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 podman[53400]: 2024-04-20 21:30:35.212650535 +0000 UTC m=+0.368013414 container init 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:30:35.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:35.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.466+0000 7f5e8be00640 1 -- 172.21.15.7:0/1780653121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1040c0 msgr2=0x7f5e8c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.466+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1780653121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c1044e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009900 tx=0x7f5e7c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 -- 172.21.15.7:0/1780653121 shutdown_connections 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1780653121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1052b0 0x7f5e8c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1780653121 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1780653121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c1044e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 -- 172.21.15.7:0/1780653121 >> 172.21.15.7:0/1780653121 conn(0x7f5e8c0ff630 msgr2=0x7f5e8c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 -- 172.21.15.7:0/1780653121 shutdown_connections 2024-04-20T21:30:35.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.467+0000 7f5e8be00640 1 -- 172.21.15.7:0/1780653121 wait complete. 2024-04-20T21:30:35.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 podman[53400]: 2024-04-20 21:30:35.215270345 +0000 UTC m=+0.370633221 container start 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-20T21:30:35.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mds[53419]: -- 172.21.15.186:0/3129817792 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558901a2da40 con 0x558901a01000 2024-04-20T21:30:35.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: starting mds.i at 2024-04-20T21:30:35.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 bash[53400]: 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 2024-04-20T21:30:35.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:30:35 smithi186 systemd[1]: Started Ceph mds.i for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:35.468 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:35 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 Processor -- start 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 -- start start 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1052b0 0x7f5e8c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10fd70 con 0x7f5e8c1052b0 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10fbf0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c10fef0 con 0x7f5e8c105c70 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1052b0 0x7f5e8c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:55748/0 (socket says 172.21.15.7:55748) 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8aa00640 1 -- 172.21.15.7:0/1502349162 learned_addr learned my addr 172.21.15.7:0/1502349162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.468+0000 7f5e8b400640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 -- 172.21.15.7:0/1502349162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 msgr2=0x7f5e8c1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 -- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1052b0 msgr2=0x7f5e8c19a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1052b0 0x7f5e8c19a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 -- 172.21.15.7:0/1502349162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c1a1fe0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8b400640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e8aa00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.469+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c043020 con 0x7f5e8c1040c0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.470+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e7c0054e0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.470+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c1a2270 con 0x7f5e8c1040c0 2024-04-20T21:30:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.470+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c003c70 con 0x7f5e8c1040c0 2024-04-20T21:30:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.470+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a2790 con 0x7f5e8c1040c0 2024-04-20T21:30:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.471+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c10b3e0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.473+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f5e7c046020 con 0x7f5e8c1040c0 2024-04-20T21:30:35.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.474+0000 7f5e83400640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 0x7f5e700a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:35.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.474+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e7c0f20a0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.474+0000 7f5e8a000640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 0x7f5e700a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:35.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.475+0000 7f5e8a000640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 0x7f5e700a6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c19b8d0 tx=0x7f5e7800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.478+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e7c04d020 con 0x7f5e8c1040c0 2024-04-20T21:30:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.730+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e8c02d5d0 con 0x7f5e8c1040c0 2024-04-20T21:30:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.731+0000 7f5e83400640 1 -- 172.21.15.7:0/1502349162 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f5e7c0bee80 con 0x7f5e8c1040c0 2024-04-20T21:30:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:35.730 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:30:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 msgr2=0x7f5e700a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 0x7f5e700a6060 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c19b8d0 tx=0x7f5e7800a2d0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 msgr2=0x7f5e8c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000c00 tx=0x7f5e7c02c6c0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 shutdown_connections 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5e700a3ba0 0x7f5e700a6060 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e8c105c70 0x7f5e8c1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e8c1052b0 0x7f5e8c19a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 --2- 172.21.15.7:0/1502349162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c1040c0 0x7f5e8c19a410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 >> 172.21.15.7:0/1502349162 conn(0x7f5e8c0ff630 msgr2=0x7f5e8c1085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 shutdown_connections 2024-04-20T21:30:35.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:35.735+0000 7f5e8be00640 1 -- 172.21.15.7:0/1502349162 wait complete. 2024-04-20T21:30:35.751 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:35.752 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:35 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:35.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:35 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:36 smithi186 ceph-mon[22157]: Deploying daemon mds.g on smithi007 2024-04-20T21:30:36.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:36 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:boot 2024-04-20T21:30:36.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:36 smithi186 ceph-mon[22157]: fsmap 9 up:standby 2024-04-20T21:30:36.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:36 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:30:36.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:36 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1502349162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:36.746 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:36 smithi007 ceph-mon[16867]: Deploying daemon mds.g on smithi007 2024-04-20T21:30:36.747 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:36 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:boot 2024-04-20T21:30:36.747 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:36 smithi007 ceph-mon[16867]: fsmap 9 up:standby 2024-04-20T21:30:36.747 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:36 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:30:36.747 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1502349162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:36.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:36 smithi016 ceph-mon[21409]: Deploying daemon mds.g on smithi007 2024-04-20T21:30:36.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:36 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:boot 2024-04-20T21:30:36.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:36 smithi016 ceph-mon[21409]: fsmap 9 up:standby 2024-04-20T21:30:36.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:36 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:30:36.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:36 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1502349162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:37.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:37.770 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:37 smithi007 ceph-mon[16867]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:37.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:37 smithi016 ceph-mon[21409]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:37.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:37 smithi186 ceph-mon[22157]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.247+0000 7fe90f800640 1 -- 172.21.15.7:0/3828487329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908077b60 msgr2=0x7fe908077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 --2- 172.21.15.7:0/3828487329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908077b60 0x7fe908077fc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009900 tx=0x7fe8f402bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 -- 172.21.15.7:0/3828487329 shutdown_connections 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 --2- 172.21.15.7:0/3828487329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908077b60 0x7fe908077fc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 --2- 172.21.15.7:0/3828487329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908078500 0x7fe908078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 --2- 172.21.15.7:0/3828487329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908076920 0x7fe908076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 -- 172.21.15.7:0/3828487329 >> 172.21.15.7:0/3828487329 conn(0x7fe9080ff750 msgr2=0x7fe908101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 -- 172.21.15.7:0/3828487329 shutdown_connections 2024-04-20T21:30:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.248+0000 7fe90f800640 1 -- 172.21.15.7:0/3828487329 wait complete. 2024-04-20T21:30:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.250+0000 7fe90f800640 1 Processor -- start 2024-04-20T21:30:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.250+0000 7fe90f800640 1 -- start start 2024-04-20T21:30:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.251+0000 7fe90f800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 0x7fe908111f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.251+0000 7fe90f800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 0x7fe90810e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:38.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.251+0000 7fe90f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908078500 0x7fe90810efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe90f800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9081142f0 con 0x7fe908076920 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe90f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe908114170 con 0x7fe908078500 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe90f800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe908114470 con 0x7fe908077b60 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe907e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 0x7fe90810e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe907e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 0x7fe90810e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44646/0 (socket says 172.21.15.7:44646) 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe907e00640 1 -- 172.21.15.7:0/1511408882 learned_addr learned my addr 172.21.15.7:0/1511408882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe90ce00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 0x7fe908111f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:38.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.252+0000 7fe90d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908078500 0x7fe90810efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.253+0000 7fe90ce00640 1 -- 172.21.15.7:0/1511408882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 msgr2=0x7fe90810e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.253+0000 7fe90ce00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 0x7fe90810e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.253+0000 7fe90ce00640 1 -- 172.21.15.7:0/1511408882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908078500 msgr2=0x7fe90810efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.253+0000 7fe90ce00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908078500 0x7fe90810efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.253+0000 7fe90ce00640 1 -- 172.21.15.7:0/1511408882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9081aeb30 con 0x7fe908076920 2024-04-20T21:30:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.254+0000 7fe90ce00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 0x7fe908111f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00d930 tx=0x7fe8fc00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.255+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc017070 con 0x7fe908076920 2024-04-20T21:30:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.255+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9081aee20 con 0x7fe908076920 2024-04-20T21:30:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.256+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9081af290 con 0x7fe908076920 2024-04-20T21:30:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.259+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe908108340 con 0x7fe908076920 2024-04-20T21:30:38.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.262+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8fc010460 con 0x7fe908076920 2024-04-20T21:30:38.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.262+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc020060 con 0x7fe908076920 2024-04-20T21:30:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.265+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe8fc00f6a0 con 0x7fe908076920 2024-04-20T21:30:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.268+0000 7fe905600640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 0x7fe8f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.268+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8fc0cf870 con 0x7fe908076920 2024-04-20T21:30:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.270+0000 7fe907e00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 0x7fe8f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.273+0000 7fe907e00640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 0x7fe8f00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40096c0 tx=0x7fe8f4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.276+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe8fc09c000 con 0x7fe908076920 2024-04-20T21:30:38.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.535+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe90802d810 con 0x7fe908076920 2024-04-20T21:30:38.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.536+0000 7fe905600640 1 -- 172.21.15.7:0/1511408882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fe8fc0a0300 con 0x7fe908076920 2024-04-20T21:30:38.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:38.535 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:30:38.535 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 10 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 msgr2=0x7fe8f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 0x7fe8f00a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40096c0 tx=0x7fe8f4005a20 comp rx=0 tx=0).stop 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 msgr2=0x7fe908111f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 0x7fe908111f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00d930 tx=0x7fe8fc00ddf0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 shutdown_connections 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe8f00a3bb0 0x7fe8f00a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe908078500 0x7fe90810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe908077b60 0x7fe90810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 --2- 172.21.15.7:0/1511408882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe908076920 0x7fe908111f10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:38.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 >> 172.21.15.7:0/1511408882 conn(0x7fe9080ff750 msgr2=0x7fe908100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:38.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 shutdown_connections 2024-04-20T21:30:38.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:38.539+0000 7fe90f800640 1 -- 172.21.15.7:0/1511408882 wait complete. 2024-04-20T21:30:39.072 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:38 smithi007 podman[55854]: 2024-04-20 21:30:38.891693092 +0000 UTC m=+0.163073611 container create 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T21:30:39.485 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 podman[55854]: 2024-04-20 21:30:39.170013714 +0000 UTC m=+0.441394276 container init 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T21:30:39.485 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 podman[55854]: 2024-04-20 21:30:39.182266287 +0000 UTC m=+0.453646786 container start 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:30:39.485 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-mds[55879]: -- 172.21.15.7:0/1095613050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590b7bcfa40 con 0x5590b7ba3000 2024-04-20T21:30:39.486 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g[55873]: starting mds.g at 2024-04-20T21:30:39.486 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 bash[55854]: 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 2024-04-20T21:30:39.486 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:30:39 smithi007 systemd[1]: Started Ceph mds.g for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:39.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-mon[16867]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1511408882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:39.765 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:39.766 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:39 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:39.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:39 smithi016 ceph-mon[21409]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:39 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1511408882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:39 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:39 smithi186 ceph-mon[22157]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:39 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1511408882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:39 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:40.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.7:6838/766857391,v1:172.21.15.7:6839/766857391] up:boot 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: fsmap 10 up:standby 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:40.744 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:40.745 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:40 smithi007 ceph-mon[16867]: Deploying daemon mds.a on smithi007 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 -- 172.21.15.7:0/2936337197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 msgr2=0x7f237c10c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 --2- 172.21.15.7:0/2936337197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c10c500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2374009be0 tx=0x7f237402bef0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 -- 172.21.15.7:0/2936337197 shutdown_connections 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 --2- 172.21.15.7:0/2936337197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c10c500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 --2- 172.21.15.7:0/2936337197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 --2- 172.21.15.7:0/2936337197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c1044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 -- 172.21.15.7:0/2936337197 >> 172.21.15.7:0/2936337197 conn(0x7f237c0ff630 msgr2=0x7f237c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 -- 172.21.15.7:0/2936337197 shutdown_connections 2024-04-20T21:30:40.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.778+0000 7f2381200640 1 -- 172.21.15.7:0/2936337197 wait complete. 2024-04-20T21:30:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.779+0000 7f2381200640 1 Processor -- start 2024-04-20T21:30:40.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.779+0000 7f2381200640 1 -- start start 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10fda0 con 0x7f237c1052b0 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10fc20 con 0x7f237c105c70 2024-04-20T21:30:40.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2381200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f237c10ff20 con 0x7f237c1040c0 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44596/0 (socket says 172.21.15.7:44596) 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 -- 172.21.15.7:0/3222916001 learned_addr learned my addr 172.21.15.7:0/3222916001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f237b400640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f237be00640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 -- 172.21.15.7:0/3222916001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 msgr2=0x7f237c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 -- 172.21.15.7:0/3222916001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 msgr2=0x7f237c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f2373e00640 1 -- 172.21.15.7:0/3222916001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c1a1f10 con 0x7f237c1052b0 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.780+0000 7f237b400640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:40.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2373e00640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f236400e9c0 tx=0x7f236400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236400cde0 con 0x7f237c1052b0 2024-04-20T21:30:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1a2200 con 0x7f237c1052b0 2024-04-20T21:30:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23640035f0 con 0x7f237c1052b0 2024-04-20T21:30:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364010650 con 0x7f237c1052b0 2024-04-20T21:30:40.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.781+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c1a27a0 con 0x7f237c1052b0 2024-04-20T21:30:40.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.783+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c10b310 con 0x7f237c1052b0 2024-04-20T21:30:40.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.785+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f23640037a0 con 0x7f237c1052b0 2024-04-20T21:30:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.786+0000 7f2378c00640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 0x7f235c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.786+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23640d0410 con 0x7f237c1052b0 2024-04-20T21:30:40.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.786+0000 7f237b400640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 0x7f235c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.787+0000 7f237b400640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 0x7f235c0a6060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f237c105110 tx=0x7f236c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:40.789+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f236409d280 con 0x7f237c1052b0 2024-04-20T21:30:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.7:6838/766857391,v1:172.21.15.7:6839/766857391] up:boot 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: fsmap 10 up:standby 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:40 smithi016 ceph-mon[21409]: Deploying daemon mds.a on smithi007 2024-04-20T21:30:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.7:6838/766857391,v1:172.21.15.7:6839/766857391] up:boot 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: fsmap 10 up:standby 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:40 smithi186 ceph-mon[22157]: Deploying daemon mds.a on smithi007 2024-04-20T21:30:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.052+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f237c02d5d0 con 0x7f237c1052b0 2024-04-20T21:30:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.054+0000 7f2378c00640 1 -- 172.21.15.7:0/3222916001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8821 (secure 0 0 0) 0x7f236409cb80 con 0x7f237c1052b0 2024-04-20T21:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:41.053 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:30:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 11 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.056+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 msgr2=0x7f235c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.056+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 0x7f235c0a6060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f237c105110 tx=0x7f236c005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.056+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 msgr2=0x7f237c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.056+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f236400e9c0 tx=0x7f236400ee80 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 shutdown_connections 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f235c0a3ba0 0x7f235c0a6060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c105c70 0x7f237c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237c1052b0 0x7f237c19a830 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 --2- 172.21.15.7:0/3222916001 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f237c1040c0 0x7f237c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 >> 172.21.15.7:0/3222916001 conn(0x7f237c0ff630 msgr2=0x7f237c1084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 shutdown_connections 2024-04-20T21:30:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:41.057+0000 7f2381200640 1 -- 172.21.15.7:0/3222916001 wait complete. 2024-04-20T21:30:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:41 smithi007 ceph-mon[16867]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:41.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:41 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3222916001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:41.846 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:41 smithi016 ceph-mon[21409]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:41.846 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:41 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3222916001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:41.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:41 smithi186 ceph-mon[22157]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:41.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:41 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3222916001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:42.706 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:42 smithi007 systemd[1]: Starting Ceph mds.a for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:43.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.293+0000 7f229f200640 1 -- 172.21.15.7:0/33603440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981040c0 msgr2=0x7f22981044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.293+0000 7f229f200640 1 --2- 172.21.15.7:0/33603440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981040c0 0x7f22981044e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f228c009900 tx=0x7f228c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 -- 172.21.15.7:0/33603440 shutdown_connections 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 --2- 172.21.15.7:0/33603440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f229810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 --2- 172.21.15.7:0/33603440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981052b0 0x7f2298105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 --2- 172.21.15.7:0/33603440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981040c0 0x7f22981044e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 -- 172.21.15.7:0/33603440 >> 172.21.15.7:0/33603440 conn(0x7f22980ff630 msgr2=0x7f2298101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 -- 172.21.15.7:0/33603440 shutdown_connections 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.294+0000 7f229f200640 1 -- 172.21.15.7:0/33603440 wait complete. 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 Processor -- start 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 -- start start 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981040c0 0x7f229819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 0x7f229819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f229810fd50 con 0x7f22981052b0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f229810fbd0 con 0x7f2298105c70 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.295+0000 7f229f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f229810fed0 con 0x7f22981040c0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 0x7f229819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57302/0 (socket says 172.21.15.7:57302) 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229e800640 1 -- 172.21.15.7:0/2808078755 learned_addr learned my addr 172.21.15.7:0/2808078755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229de00640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981040c0 0x7f229819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 -- 172.21.15.7:0/2808078755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981040c0 msgr2=0x7f229819a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981040c0 0x7f229819a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 -- 172.21.15.7:0/2808078755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 msgr2=0x7f22981a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 -- 172.21.15.7:0/2808078755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22981a1fd0 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229e800640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.296+0000 7f229d400640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 0x7f229819a8f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2288005fd0 tx=0x7f228800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.297+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288009c30 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.297+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2288009dd0 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.297+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288010460 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.297+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22981a22c0 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.297+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22981a27c0 con 0x7f22981052b0 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.298+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229810b3e0 con 0x7f22981052b0 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.301+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2288003ef0 con 0x7f22981052b0 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.302+0000 7f2292a00640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 0x7f22800a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.302+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22880cfba0 con 0x7f22981052b0 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.303+0000 7f229de00640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 0x7f22800a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.303+0000 7f229de00640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 0x7f22800a6060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f228c000c00 tx=0x7f228c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.307+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2288017020 con 0x7f22981052b0 2024-04-20T21:30:43.342 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 podman[56206]: 2024-04-20 21:30:42.986448826 +0000 UTC m=+0.038687035 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:43.342 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 podman[56206]: 2024-04-20 21:30:43.105705951 +0000 UTC m=+0.157944148 container create f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-20T21:30:43.342 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 podman[56206]: 2024-04-20 21:30:43.342250555 +0000 UTC m=+0.394488771 container init f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:30:43.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.599+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2298062b60 con 0x7f22981052b0 2024-04-20T21:30:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.600+0000 7f2292a00640 1 -- 172.21.15.7:0/2808078755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f228809c330 con 0x7f22981052b0 2024-04-20T21:30:43.599 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:43.600 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:30:43.600 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 msgr2=0x7f22800a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 0x7f22800a6060 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f228c000c00 tx=0x7f228c005fd0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 msgr2=0x7f229819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 0x7f229819a8f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2288005fd0 tx=0x7f228800dde0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 shutdown_connections 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f22800a3ba0 0x7f22800a6060 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2298105c70 0x7f22981a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22981052b0 0x7f229819a8f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 --2- 172.21.15.7:0/2808078755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f22981040c0 0x7f229819a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.604+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 >> 172.21.15.7:0/2808078755 conn(0x7f22980ff630 msgr2=0x7f22981085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:43.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.605+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 shutdown_connections 2024-04-20T21:30:43.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:43.605+0000 7f229f200640 1 -- 172.21.15.7:0/2808078755 wait complete. 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:43 smithi007 ceph-mon[16867]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 podman[56206]: 2024-04-20 21:30:43.355211129 +0000 UTC m=+0.407449325 container start f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 ceph-mds[56264]: -- 172.21.15.7:0/2010100767 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5bbeffa40 con 0x55f5bbed2800 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a[56241]: starting mds.a at 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 bash[56206]: f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed 2024-04-20T21:30:43.622 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:30:43 smithi007 systemd[1]: Started Ceph mds.a for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:43 smithi016 ceph-mon[21409]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:43.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:43 smithi186 ceph-mon[22157]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.7:6840/771888475,v1:172.21.15.7:6841/771888475] up:boot 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: fsmap 11 up:standby 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2808078755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:44.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:44.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:44 smithi016 ceph-mon[21409]: Deploying daemon mds.b on smithi016 2024-04-20T21:30:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.7:6840/771888475,v1:172.21.15.7:6841/771888475] up:boot 2024-04-20T21:30:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: fsmap 11 up:standby 2024-04-20T21:30:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2808078755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:44.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:44 smithi186 ceph-mon[22157]: Deploying daemon mds.b on smithi016 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.7:6840/771888475,v1:172.21.15.7:6841/771888475] up:boot 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: fsmap 11 up:standby 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2808078755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:45.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-20T21:30:45.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-20T21:30:45.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:45.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:44 smithi007 ceph-mon[16867]: Deploying daemon mds.b on smithi016 2024-04-20T21:30:45.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 -- 172.21.15.7:0/1903643175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 msgr2=0x7f1050105710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 --2- 172.21.15.7:0/1903643175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f1050105710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1038009900 tx=0x7f103802bc50 comp rx=0 tx=0).stop 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 -- 172.21.15.7:0/1903643175 shutdown_connections 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 --2- 172.21.15.7:0/1903643175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f1050105710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 --2- 172.21.15.7:0/1903643175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f105010c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 --2- 172.21.15.7:0/1903643175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f10501044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.856+0000 7f1055000640 1 -- 172.21.15.7:0/1903643175 >> 172.21.15.7:0/1903643175 conn(0x7f10500ff630 msgr2=0x7f1050101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.857+0000 7f1055000640 1 -- 172.21.15.7:0/1903643175 shutdown_connections 2024-04-20T21:30:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.857+0000 7f1055000640 1 -- 172.21.15.7:0/1903643175 wait complete. 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 Processor -- start 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 -- start start 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f105019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f105019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f10501a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f105010fd30 con 0x7f1050105c50 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f105010fbb0 con 0x7f1050105290 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.858+0000 7f1055000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f105010feb0 con 0x7f10501040a0 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f105019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f105019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:38412/0 (socket says 172.21.15.7:38412) 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104f400640 1 -- 172.21.15.7:0/646422154 learned_addr learned my addr 172.21.15.7:0/646422154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f10501a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104ea00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f105019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 -- 172.21.15.7:0/646422154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 msgr2=0x7f105019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f105019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 -- 172.21.15.7:0/646422154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 msgr2=0x7f105019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f105019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.859+0000 7f104fe00640 1 -- 172.21.15.7:0/646422154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501a2050 con 0x7f1050105c50 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f104fe00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f10501a19e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f104800e970 tx=0x7f104800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800ccc0 con 0x7f1050105c50 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10480035f0 con 0x7f1050105c50 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104800ce60 con 0x7f1050105c50 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501a2340 con 0x7f1050105c50 2024-04-20T21:30:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.860+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501a2810 con 0x7f1050105c50 2024-04-20T21:30:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.861+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105010b530 con 0x7f1050105c50 2024-04-20T21:30:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.864+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f10480037a0 con 0x7f1050105c50 2024-04-20T21:30:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.865+0000 7f1047e00640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 0x7f10340a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.866+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f10480d02f0 con 0x7f1050105c50 2024-04-20T21:30:45.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.866+0000 7f104f400640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 0x7f10340a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:45.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.867+0000 7f104f400640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 0x7f10340a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1040009fa0 tx=0x7f10400096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:45.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:45.868+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f104809d160 con 0x7f1050105c50 2024-04-20T21:30:45.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:45 smithi016 ceph-mon[21409]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:45.945 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:45 smithi016 systemd[1]: Starting Ceph mds.b for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:30:45.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:45 smithi186 ceph-mon[22157]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:46.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:45 smithi007 ceph-mon[16867]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:46.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.124+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f105002d5d0 con 0x7f1050105c50 2024-04-20T21:30:46.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.126+0000 7f1047e00640 1 -- 172.21.15.7:0/646422154 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9641 (secure 0 0 0) 0x7f104809ca60 con 0x7f1050105c50 2024-04-20T21:30:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:46.125 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-20T21:30:46.125 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 msgr2=0x7f10340a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 0x7f10340a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1040009fa0 tx=0x7f10400096d0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 msgr2=0x7f10501a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f10501a19e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f104800e970 tx=0x7f104800ee30 comp rx=0 tx=0).stop 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 shutdown_connections 2024-04-20T21:30:46.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f10340a3ba0 0x7f10340a6060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1050105290 0x7f105019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1050105c50 0x7f10501a19e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 --2- 172.21.15.7:0/646422154 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10501040a0 0x7f105019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 >> 172.21.15.7:0/646422154 conn(0x7f10500ff630 msgr2=0x7f1050108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 shutdown_connections 2024-04-20T21:30:46.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:46.129+0000 7f1055000640 1 -- 172.21.15.7:0/646422154 wait complete. 2024-04-20T21:30:46.297 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 podman[51685]: 2024-04-20 21:30:45.938000215 +0000 UTC m=+0.018475849 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:30:46.297 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 podman[51685]: 2024-04-20 21:30:46.161535135 +0000 UTC m=+0.242010737 container create 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 podman[51685]: 2024-04-20 21:30:46.4062012 +0000 UTC m=+0.486676803 container init 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 podman[51685]: 2024-04-20 21:30:46.408894249 +0000 UTC m=+0.489369852 container start 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mds[51705]: -- 172.21.15.16:0/2666994710 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557070f93a40 con 0x557070f67000 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: starting mds.b at 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 bash[51685]: 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b 2024-04-20T21:30:46.627 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:46 smithi016 systemd[1]: Started Ceph mds.b for 8e9da876-ff5b-11ee-bc93-c7b262605968. 2024-04-20T21:30:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:46 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/646422154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:46 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/646422154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:46.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:46 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:46 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/646422154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:47.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:46 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.900 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:boot 2024-04-20T21:30:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: fsmap 12 up:standby 2024-04-20T21:30:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:30:47.901 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:47 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:47.930 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:47.930 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:47.931 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:boot 2024-04-20T21:30:47.931 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: fsmap 12 up:standby 2024-04-20T21:30:47.931 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:30:47.931 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:47 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:boot 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: fsmap 12 up:standby 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:30:48.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:47 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:48.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 -- 172.21.15.7:0/1732645514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c101720 msgr2=0x7fc94c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 --2- 172.21.15.7:0/1732645514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c101720 0x7fc94c101ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc934009900 tx=0x7fc93402bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 -- 172.21.15.7:0/1732645514 shutdown_connections 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 --2- 172.21.15.7:0/1732645514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c1020e0 0x7fc94c108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 --2- 172.21.15.7:0/1732645514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c101720 0x7fc94c101ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 --2- 172.21.15.7:0/1732645514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 0x7fc94c100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 -- 172.21.15.7:0/1732645514 >> 172.21.15.7:0/1732645514 conn(0x7fc94c0fbaa0 msgr2=0x7fc94c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 -- 172.21.15.7:0/1732645514 shutdown_connections 2024-04-20T21:30:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.376+0000 7fc94b200640 1 -- 172.21.15.7:0/1732645514 wait complete. 2024-04-20T21:30:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 Processor -- start 2024-04-20T21:30:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 -- start start 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 0x7fc94c109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c1020e0 0x7fc94c1a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10ca90 con 0x7fc94c100530 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10c910 con 0x7fc94c101720 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc94b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10cc10 con 0x7fc94c1020e0 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.378+0000 7fc941400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57348/0 (socket says 172.21.15.7:57348) 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 -- 172.21.15.7:0/243659591 learned_addr learned my addr 172.21.15.7:0/243659591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc94a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c1020e0 0x7fc94c1a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc949e00640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 0x7fc94c109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 -- 172.21.15.7:0/243659591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c1020e0 msgr2=0x7fc94c1a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c1020e0 0x7fc94c1a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 -- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 msgr2=0x7fc94c109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 0x7fc94c109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 -- 172.21.15.7:0/243659591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c1a1fe0 con 0x7fc94c101720 2024-04-20T21:30:48.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.379+0000 7fc941400640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc93402c8f0 tx=0x7fc93402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.380+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934041070 con 0x7fc94c101720 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.380+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc934004450 con 0x7fc94c101720 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.380+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934034440 con 0x7fc94c101720 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.380+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c1a2210 con 0x7fc94c101720 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.380+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1a26b0 con 0x7fc94c101720 2024-04-20T21:30:48.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.381+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c107780 con 0x7fc94c101720 2024-04-20T21:30:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.383+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fc934044070 con 0x7fc94c101720 2024-04-20T21:30:48.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.385+0000 7fc943400640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 0x7fc9300a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.385+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc9340f1990 con 0x7fc94c101720 2024-04-20T21:30:48.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.385+0000 7fc949e00640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 0x7fc9300a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.386+0000 7fc949e00640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 0x7fc9300a6110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc94c101580 tx=0x7fc93c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.390+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc9340be1c0 con 0x7fc94c101720 2024-04-20T21:30:48.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.643+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc94c02d820 con 0x7fc94c101720 2024-04-20T21:30:48.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.651+0000 7fc943400640 1 -- 172.21.15.7:0/243659591 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7fc9340c24c0 con 0x7fc94c101720 2024-04-20T21:30:48.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:48.651 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24523,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24559,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-20T21:30:48.651 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.653+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 msgr2=0x7fc9300a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.653+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 0x7fc9300a6110 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc94c101580 tx=0x7fc93c005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 msgr2=0x7fc94c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc93402c8f0 tx=0x7fc93402c660 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 shutdown_connections 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc9300a3c50 0x7fc9300a6110 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc94c101720 0x7fc94c19f2a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc94c1020e0 0x7fc94c1a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 --2- 172.21.15.7:0/243659591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94c100530 0x7fc94c109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 >> 172.21.15.7:0/243659591 conn(0x7fc94c0fbaa0 msgr2=0x7fc94c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 shutdown_connections 2024-04-20T21:30:48.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.654+0000 7fc94b200640 1 -- 172.21.15.7:0/243659591 wait complete. 2024-04-20T21:30:48.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.891+0000 7fe252400640 1 -- 172.21.15.7:0/2801012060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c077b60 msgr2=0x7fe24c077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.891+0000 7fe252400640 1 --2- 172.21.15.7:0/2801012060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c077b60 0x7fe24c077fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe238009900 tx=0x7fe23802bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 -- 172.21.15.7:0/2801012060 shutdown_connections 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 --2- 172.21.15.7:0/2801012060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c077b60 0x7fe24c077fc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 --2- 172.21.15.7:0/2801012060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c078500 0x7fe24c078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 --2- 172.21.15.7:0/2801012060 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 -- 172.21.15.7:0/2801012060 >> 172.21.15.7:0/2801012060 conn(0x7fe24c0ff630 msgr2=0x7fe24c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 -- 172.21.15.7:0/2801012060 shutdown_connections 2024-04-20T21:30:48.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.892+0000 7fe252400640 1 -- 172.21.15.7:0/2801012060 wait complete. 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 Processor -- start 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 -- start start 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c078500 0x7fe24c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c1a3f10 0x7fe24c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c114190 con 0x7fe24c1a3f10 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c114010 con 0x7fe24c078500 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe252400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c114310 con 0x7fe24c076920 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe251000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe251000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:38462/0 (socket says 172.21.15.7:38462) 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe251000640 1 -- 172.21.15.7:0/3199748992 learned_addr learned my addr 172.21.15.7:0/3199748992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.893+0000 7fe24be00640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c078500 0x7fe24c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251a00640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c1a3f10 0x7fe24c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 -- 172.21.15.7:0/3199748992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c078500 msgr2=0x7fe24c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c078500 0x7fe24c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 -- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c1a3f10 msgr2=0x7fe24c1a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c1a3f10 0x7fe24c1a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 -- 172.21.15.7:0/3199748992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1a6970 con 0x7fe24c076920 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe251000640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe24000e9f0 tx=0x7fe24000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24000cd80 con 0x7fe24c076920 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe240010910 con 0x7fe24c076920 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.894+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1a6c60 con 0x7fe24c076920 2024-04-20T21:30:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.895+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240018990 con 0x7fe24c076920 2024-04-20T21:30:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.895+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c1a7180 con 0x7fe24c076920 2024-04-20T21:30:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.896+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c110a30 con 0x7fe24c076920 2024-04-20T21:30:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.898+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe240010ac0 con 0x7fe24c076920 2024-04-20T21:30:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.899+0000 7fe249600640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 0x7fe2340a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.899+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe2400d03c0 con 0x7fe24c076920 2024-04-20T21:30:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.900+0000 7fe24be00640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 0x7fe2340a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.900+0000 7fe24be00640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 0x7fe2340a6060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe238009230 tx=0x7fe23802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:48.908+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe24009d1a0 con 0x7fe24c076920 2024-04-20T21:30:49.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.127+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe24c062b60 con 0x7fe24c076920 2024-04-20T21:30:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.128+0000 7fe249600640 1 -- 172.21.15.7:0/3199748992 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18340 (secure 0 0 0) 0x7fe2400bd0e0 con 0x7fe24c076920 2024-04-20T21:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:49.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:29:55.003374+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 msgr2=0x7fe2340a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 0x7fe2340a6060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe238009230 tx=0x7fe23802df90 comp rx=0 tx=0).stop 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 msgr2=0x7fe24c19e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe24000e9f0 tx=0x7fe24000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 shutdown_connections 2024-04-20T21:30:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2340a3ba0 0x7fe2340a6060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe24c078500 0x7fe24c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe24c1a3f10 0x7fe24c1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 --2- 172.21.15.7:0/3199748992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe24c076920 0x7fe24c19e9e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 >> 172.21.15.7:0/3199748992 conn(0x7fe24c0ff630 msgr2=0x7fe24c101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 shutdown_connections 2024-04-20T21:30:49.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.132+0000 7fe252400640 1 -- 172.21.15.7:0/3199748992 wait complete. 2024-04-20T21:30:49.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:49.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 -- 172.21.15.7:0/41366965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0076920 msgr2=0x7f8fb0076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/41366965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0076920 0x7f8fb0076d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009be0 tx=0x7f8fa002bef0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 -- 172.21.15.7:0/41366965 shutdown_connections 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/41366965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0077b60 0x7f8fb0077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/41366965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0078500 0x7f8fb0078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/41366965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0076920 0x7f8fb0076d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 -- 172.21.15.7:0/41366965 >> 172.21.15.7:0/41366965 conn(0x7f8fb00ff6e0 msgr2=0x7f8fb0101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.355+0000 7f8fb6a00640 1 -- 172.21.15.7:0/41366965 shutdown_connections 2024-04-20T21:30:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.356+0000 7f8fb6a00640 1 -- 172.21.15.7:0/41366965 wait complete. 2024-04-20T21:30:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.356+0000 7f8fb6a00640 1 Processor -- start 2024-04-20T21:30:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 -- start start 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0078500 0x7f8fb01a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0114240 con 0x7f8fb0076920 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01140c0 con 0x7f8fb0078500 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb6a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb01143c0 con 0x7f8fb0077b60 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fa7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fa7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:38480/0 (socket says 172.21.15.7:38480) 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fb4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0078500 0x7f8fb01a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41322/0 (socket says 172.21.15.7:41322) 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.357+0000 7f8fa7400640 1 -- 172.21.15.7:0/2922730499 learned_addr learned my addr 172.21.15.7:0/2922730499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:49.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 -- 172.21.15.7:0/2922730499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0078500 msgr2=0x7f8fb01a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0078500 0x7f8fb01a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 -- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 msgr2=0x7f8fb019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 -- 172.21.15.7:0/2922730499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01a6490 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fafe00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fa7400640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9f0 tx=0x7f8f9800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9800ccc0 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f98010910 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a6780 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.358+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f980188a0 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.359+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a6d20 con 0x7f8fb0077b60 2024-04-20T21:30:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.360+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0110cb0 con 0x7f8fb0077b60 2024-04-20T21:30:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.361+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f8f98010430 con 0x7f8fb0077b60 2024-04-20T21:30:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.363+0000 7f8fad600640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 0x7f8f940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.363+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f980d03d0 con 0x7f8fb0077b60 2024-04-20T21:30:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.363+0000 7f8fafe00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 0x7f8f940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.364+0000 7f8fafe00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 0x7f8f940a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0006010 tx=0x7f8fa0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.367+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8f9809d1f0 con 0x7f8fb0077b60 2024-04-20T21:30:49.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/243659591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:49 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3199748992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.624+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fb002d5d0 con 0x7f8fb0077b60 2024-04-20T21:30:49.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.624+0000 7f8fad600640 1 -- 172.21.15.7:0/2922730499 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8f9809caf0 con 0x7f8fb0077b60 2024-04-20T21:30:49.624 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 msgr2=0x7f8f940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 0x7f8f940a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0006010 tx=0x7f8fa0000f40 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 msgr2=0x7f8fb019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800e9f0 tx=0x7f8f9800edf0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 shutdown_connections 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8f940a3c30 0x7f8f940a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fb0078500 0x7f8fb01a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8fb0077b60 0x7f8fb019edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 --2- 172.21.15.7:0/2922730499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fb0076920 0x7f8fb019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 >> 172.21.15.7:0/2922730499 conn(0x7f8fb00ff6e0 msgr2=0x7f8fb0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 shutdown_connections 2024-04-20T21:30:49.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.627+0000 7f8fb6a00640 1 -- 172.21.15.7:0/2922730499 wait complete. 2024-04-20T21:30:49.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:30:49.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/243659591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:49 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3199748992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/243659591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:49.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:49 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3199748992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 -- 172.21.15.7:0/1275432658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 msgr2=0x7f21a41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 --2- 172.21.15.7:0/1275432658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a41034c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2190009900 tx=0x7f219002bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 -- 172.21.15.7:0/1275432658 shutdown_connections 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 --2- 172.21.15.7:0/1275432658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 0x7f21a410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 --2- 172.21.15.7:0/1275432658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a41034c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 --2- 172.21.15.7:0/1275432658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 -- 172.21.15.7:0/1275432658 >> 172.21.15.7:0/1275432658 conn(0x7f21a40fd420 msgr2=0x7f21a40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 -- 172.21.15.7:0/1275432658 shutdown_connections 2024-04-20T21:30:49.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.854+0000 7f21a8a00640 1 -- 172.21.15.7:0/1275432658 wait complete. 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.855+0000 7f21a8a00640 1 Processor -- start 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.855+0000 7f21a8a00640 1 -- start start 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 0x7f21a41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410db90 con 0x7f21a4103040 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410da10 con 0x7f21a4101e60 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a8a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410dd10 con 0x7f21a4103a00 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 0x7f21a41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57414/0 (socket says 172.21.15.7:57414) 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a3400640 1 -- 172.21.15.7:0/2964059282 learned_addr learned my addr 172.21.15.7:0/2964059282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.856+0000 7f21a2a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 -- 172.21.15.7:0/2964059282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 msgr2=0x7f21a41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 0x7f21a41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 -- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 msgr2=0x7f21a419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 -- 172.21.15.7:0/2964059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a41a1f80 con 0x7f21a4101e60 2024-04-20T21:30:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a3400640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f219400e9c0 tx=0x7f219400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400cd80 con 0x7f21a4101e60 2024-04-20T21:30:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21940035f0 con 0x7f21a4101e60 2024-04-20T21:30:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a41a2270 con 0x7f21a4101e60 2024-04-20T21:30:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.857+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194010690 con 0x7f21a4101e60 2024-04-20T21:30:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.858+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41a2790 con 0x7f21a4101e60 2024-04-20T21:30:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.859+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a41a2a50 con 0x7f21a4101e60 2024-04-20T21:30:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.862+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f2194004020 con 0x7f21a4101e60 2024-04-20T21:30:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.863+0000 7f219be00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 0x7f21880a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.863+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f21940d04a0 con 0x7f21a4101e60 2024-04-20T21:30:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.863+0000 7f21a2a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 0x7f21880a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.864+0000 7f21a2a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 0x7f21880a6060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f219002c240 tx=0x7f219002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:49.867+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f219409d280 con 0x7f21a4101e60 2024-04-20T21:30:50.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.104+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f21a41a3140 con 0x7f21a4101e60 2024-04-20T21:30:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.105+0000 7f219be00640 1 -- 172.21.15.7:0/2964059282 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f219409cb80 con 0x7f21a4101e60 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:30:50.105 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-20T21:30:50.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.107+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 msgr2=0x7f21880a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.107+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 0x7f21880a6060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f219002c240 tx=0x7f219002df90 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.107+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 msgr2=0x7f21a419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.107+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f219400e9c0 tx=0x7f219400ee80 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 shutdown_connections 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f21880a3ba0 0x7f21880a6060 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a4103a00 0x7f21a41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21a4103040 0x7f21a419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 --2- 172.21.15.7:0/2964059282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a4101e60 0x7f21a419a360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 >> 172.21.15.7:0/2964059282 conn(0x7f21a40fd420 msgr2=0x7f21a4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 shutdown_connections 2024-04-20T21:30:50.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.108+0000 7f21a8a00640 1 -- 172.21.15.7:0/2964059282 wait complete. 2024-04-20T21:30:50.127 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:30:50.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 -- 172.21.15.7:0/611595548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f501031a0 msgr2=0x7f0f50103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 --2- 172.21.15.7:0/611595548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f501031a0 0x7f0f50103620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 -- 172.21.15.7:0/611595548 shutdown_connections 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 --2- 172.21.15.7:0/611595548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f501031a0 0x7f0f50103620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 --2- 172.21.15.7:0/611595548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50103b60 0x7f0f5010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 --2- 172.21.15.7:0/611595548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f50101fb0 0x7f0f501023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 -- 172.21.15.7:0/611595548 >> 172.21.15.7:0/611595548 conn(0x7f0f500fd520 msgr2=0x7f0f500ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 -- 172.21.15.7:0/611595548 shutdown_connections 2024-04-20T21:30:50.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.320+0000 7f0f55000640 1 -- 172.21.15.7:0/611595548 wait complete. 2024-04-20T21:30:50.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.321+0000 7f0f55000640 1 Processor -- start 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 -- start start 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 0x7f0f5019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 0x7f0f501a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f50075330 con 0x7f0f501031a0 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f500751b0 con 0x7f0f50103b60 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f55000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0f500754b0 con 0x7f0f50101fb0 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 0x7f0f5019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41366/0 (socket says 172.21.15.7:41366) 2024-04-20T21:30:50.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 0x7f0f5019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:38532/0 (socket says 172.21.15.7:38532) 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4d400640 1 -- 172.21.15.7:0/1167231868 learned_addr learned my addr 172.21.15.7:0/1167231868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.322+0000 7f0f4e800640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 0x7f0f501a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 -- 172.21.15.7:0/1167231868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 msgr2=0x7f0f5019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 0x7f0f5019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 -- 172.21.15.7:0/1167231868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 msgr2=0x7f0f501a1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 0x7f0f501a1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 -- 172.21.15.7:0/1167231868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f501a2160 con 0x7f0f501031a0 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4e800640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 0x7f0f501a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f4d400640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c043020 con 0x7f0f501031a0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f501a23f0 con 0x7f0f501031a0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f3c0054e0 con 0x7f0f501031a0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.323+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c003c40 con 0x7f0f501031a0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.324+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f501a2990 con 0x7f0f501031a0 2024-04-20T21:30:50.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.325+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f50108ba0 con 0x7f0f501031a0 2024-04-20T21:30:50.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.327+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f0f3c046020 con 0x7f0f501031a0 2024-04-20T21:30:50.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.329+0000 7f0f46a00640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 0x7f0f340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:50.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.329+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0f3c0f2390 con 0x7f0f501031a0 2024-04-20T21:30:50.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.329+0000 7f0f4de00640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 0x7f0f340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:50.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.330+0000 7f0f4de00640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 0x7f0f340a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f50103000 tx=0x7f0f40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:50.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.336+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0f3c04d020 con 0x7f0f501031a0 2024-04-20T21:30:50.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:50.551+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0f5002d5d0 con 0x7f0f501031a0 2024-04-20T21:30:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2922730499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2964059282' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:30:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2922730499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2964059282' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:30:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2922730499' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2964059282' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-20T21:30:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:30:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.358+0000 7f0f46a00640 1 -- 172.21.15.7:0/1167231868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f0f3c0bf170 con 0x7f0f501031a0 2024-04-20T21:30:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-20T21:30:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 msgr2=0x7f0f340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 0x7f0f340a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f50103000 tx=0x7f0f40005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 msgr2=0x7f0f5019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c02c650 comp rx=0 tx=0).stop 2024-04-20T21:30:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 shutdown_connections 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f0f340a3c30 0x7f0f340a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f50103b60 0x7f0f501a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f501031a0 0x7f0f5019aa80 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c02c650 comp rx=0 tx=0).stop 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 --2- 172.21.15.7:0/1167231868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f50101fb0 0x7f0f5019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.364+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 >> 172.21.15.7:0/1167231868 conn(0x7f0f500fd520 msgr2=0x7f0f500feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.365+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 shutdown_connections 2024-04-20T21:30:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.365+0000 7f0f55000640 1 -- 172.21.15.7:0/1167231868 wait complete. 2024-04-20T21:30:51.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 -- 172.21.15.7:0/2024666435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c101ec0 msgr2=0x7f555c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 --2- 172.21.15.7:0/2024666435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c101ec0 0x7f555c102340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5548009900 tx=0x7f554802bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 -- 172.21.15.7:0/2024666435 shutdown_connections 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 --2- 172.21.15.7:0/2024666435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c101ec0 0x7f555c102340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 --2- 172.21.15.7:0/2024666435 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c102880 0x7f555c109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 --2- 172.21.15.7:0/2024666435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c100cd0 0x7f555c1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 -- 172.21.15.7:0/2024666435 >> 172.21.15.7:0/2024666435 conn(0x7f555c0fc240 msgr2=0x7f555c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:51.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 -- 172.21.15.7:0/2024666435 shutdown_connections 2024-04-20T21:30:51.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.594+0000 7f5562800640 1 -- 172.21.15.7:0/2024666435 wait complete. 2024-04-20T21:30:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.595+0000 7f5562800640 1 Processor -- start 2024-04-20T21:30:51.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.595+0000 7f5562800640 1 -- start start 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.595+0000 7f5562800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5562800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c101ec0 0x7f555c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5562800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c102880 0x7f555c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5562800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f555c10dad0 con 0x7f555c102880 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5562800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f555c10d950 con 0x7f555c100cd0 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5562800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f555c10dc50 con 0x7f555c101ec0 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57472/0 (socket says 172.21.15.7:57472) 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 -- 172.21.15.7:0/954414019 learned_addr learned my addr 172.21.15.7:0/954414019 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5560a00640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c101ec0 0x7f555c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561e00640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c102880 0x7f555c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 -- 172.21.15.7:0/954414019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c101ec0 msgr2=0x7f555c19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:51.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c101ec0 0x7f555c19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 -- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c102880 msgr2=0x7f555c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c102880 0x7f555c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.596+0000 7f5561400640 1 -- 172.21.15.7:0/954414019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555c1a1f10 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f5561400640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9c0 tx=0x7f555000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555000cd80 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55500035f0 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555c1a2200 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550010690 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.597+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f555c1a2770 con 0x7f555c100cd0 2024-04-20T21:30:51.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.598+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555c1a2a30 con 0x7f555c100cd0 2024-04-20T21:30:51.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.601+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f55500037a0 con 0x7f555c100cd0 2024-04-20T21:30:51.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.603+0000 7f555a000640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 0x7f55440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:51.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.603+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f55500d04d0 con 0x7f555c100cd0 2024-04-20T21:30:51.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.603+0000 7f5560a00640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 0x7f55440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:51.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.604+0000 7f5560a00640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 0x7f55440a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f554802c240 tx=0x7f554802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:51.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.607+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f555009d140 con 0x7f555c100cd0 2024-04-20T21:30:51.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:51 smithi016 ceph-mon[21409]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:30:51.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:51 smithi186 ceph-mon[22157]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:51 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:30:51.775 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:51 smithi007 ceph-mon[16867]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:51.776 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:51 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-20T21:30:51.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:51.836+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f555c1a3260 con 0x7f555c100cd0 2024-04-20T21:30:52.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.367+0000 7f555a000640 1 -- 172.21.15.7:0/954414019 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f555009ca40 con 0x7f555c100cd0 2024-04-20T21:30:52.371 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-20T21:30:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.378+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 msgr2=0x7f55440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:52.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.378+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 0x7f55440a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f554802c240 tx=0x7f554802df90 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.378+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 msgr2=0x7f555c19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.378+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9c0 tx=0x7f555000ee80 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 shutdown_connections 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f55440a3c30 0x7f55440a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c102880 0x7f555c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f555c101ec0 0x7f555c19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 --2- 172.21.15.7:0/954414019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f555c100cd0 0x7f555c19a2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 >> 172.21.15.7:0/954414019 conn(0x7f555c0fc240 msgr2=0x7f555c109ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 shutdown_connections 2024-04-20T21:30:52.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.379+0000 7f5562800640 1 -- 172.21.15.7:0/954414019 wait complete. 2024-04-20T21:30:52.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 -- 172.21.15.7:0/1608422165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c102850 msgr2=0x7f046c104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 --2- 172.21.15.7:0/1608422165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c102850 0x7f046c104c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f045c009860 tx=0x7f045c02bb70 comp rx=0 tx=0).stop 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 -- 172.21.15.7:0/1608422165 shutdown_connections 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 --2- 172.21.15.7:0/1608422165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 --2- 172.21.15.7:0/1608422165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c105180 0x7f046c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 --2- 172.21.15.7:0/1608422165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c102850 0x7f046c104c40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 -- 172.21.15.7:0/1608422165 >> 172.21.15.7:0/1608422165 conn(0x7f046c0fc240 msgr2=0x7f046c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:52.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 -- 172.21.15.7:0/1608422165 shutdown_connections 2024-04-20T21:30:52.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.593+0000 7f046be00640 1 -- 172.21.15.7:0/1608422165 wait complete. 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.594+0000 7f046be00640 1 Processor -- start 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.594+0000 7f046be00640 1 -- start start 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 0x7f046c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:52.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10da70 con 0x7f046c102850 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10d8f0 con 0x7f046c105180 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f046c10dbf0 con 0x7f046c107ab0 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 0x7f046c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41390/0 (socket says 172.21.15.7:41390) 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 -- 172.21.15.7:0/4173939274 learned_addr learned my addr 172.21.15.7:0/4173939274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046b400640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 -- 172.21.15.7:0/4173939274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 msgr2=0x7f046c1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 -- 172.21.15.7:0/4173939274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 msgr2=0x7f046c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 0x7f046c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046aa00640 1 -- 172.21.15.7:0/4173939274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046c1a1f40 con 0x7f046c102850 2024-04-20T21:30:52.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046b400640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.595+0000 7f046a000640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 0x7f046c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f046aa00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f045c005940 tx=0x7f045c032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c041020 con 0x7f046c102850 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c02c9f0 con 0x7f046c102850 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046c1a21d0 con 0x7f046c102850 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c039040 con 0x7f046c102850 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.596+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f046c1a2710 con 0x7f046c102850 2024-04-20T21:30:52.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.598+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f046c109600 con 0x7f046c102850 2024-04-20T21:30:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.600+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f045c044020 con 0x7f046c102850 2024-04-20T21:30:52.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.601+0000 7f0470a00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 0x7f04500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:52.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.601+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f045c0f1ff0 con 0x7f046c102850 2024-04-20T21:30:52.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.602+0000 7f046a000640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 0x7f04500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.602+0000 7f046a000640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 0x7f04500a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f046c19b830 tx=0x7f0458009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.605+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f045c04b020 con 0x7f046c102850 2024-04-20T21:30:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:30:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/954414019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:52 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:30:52.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:52.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/954414019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:52 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1167231868' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: osdmap e83: 12 total, 12 up, 12 in 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/954414019' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:52 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:52.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:52.884+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f046c02d5d0 con 0x7f046c102850 2024-04-20T21:30:53.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: pgmap v315: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:30:53.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:53.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:30:53.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:53.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:53 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: pgmap v315: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:53.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:53 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: pgmap v315: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:30:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-20T21:30:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: osdmap e84: 12 total, 12 up, 12 in 2024-04-20T21:30:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-20T21:30:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-20T21:30:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-20T21:30:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: osdmap e85: 12 total, 12 up, 12 in 2024-04-20T21:30:53.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:30:53.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:30:53.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:53 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:30:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.876+0000 7f0470a00640 1 -- 172.21.15.7:0/4173939274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f045c0beb60 con 0x7f046c102850 2024-04-20T21:30:53.876 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T21:30:53.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 msgr2=0x7f04500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 0x7f04500a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f046c19b830 tx=0x7f0458009260 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 msgr2=0x7f046c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f045c005940 tx=0x7f045c032470 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 shutdown_connections 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f04500a3c30 0x7f04500a60f0 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f046c19b830 tx=0x7f0458009260 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f046c105180 0x7f046c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f046c107ab0 0x7f046c1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 --2- 172.21.15.7:0/4173939274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f046c102850 0x7f046c19a320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.881+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 >> 172.21.15.7:0/4173939274 conn(0x7f046c0fc240 msgr2=0x7f046c102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.882+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 shutdown_connections 2024-04-20T21:30:53.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:53.882+0000 7f046be00640 1 -- 172.21.15.7:0/4173939274 wait complete. 2024-04-20T21:30:53.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 -- 172.21.15.7:0/1336823386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 msgr2=0x7f6c3c108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 --2- 172.21.15.7:0/1336823386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c108920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009900 tx=0x7f6c3002bc50 comp rx=0 tx=0).stop 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 -- 172.21.15.7:0/1336823386 shutdown_connections 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 --2- 172.21.15.7:0/1336823386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c10a060 0x7f6c3c1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 --2- 172.21.15.7:0/1336823386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 --2- 172.21.15.7:0/1336823386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c108920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 -- 172.21.15.7:0/1336823386 >> 172.21.15.7:0/1336823386 conn(0x7f6c3c077450 msgr2=0x7f6c3c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 -- 172.21.15.7:0/1336823386 shutdown_connections 2024-04-20T21:30:54.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.101+0000 7f6c42a00640 1 -- 172.21.15.7:0/1336823386 wait complete. 2024-04-20T21:30:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 Processor -- start 2024-04-20T21:30:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 -- start start 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c10a060 0x7f6c3c1a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c114190 con 0x7f6c3c10a060 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c114010 con 0x7f6c3c1096a0 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.102+0000 7f6c42a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3c114310 con 0x7f6c3c108500 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c41600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45552/0 (socket says 172.21.15.7:45552) 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 -- 172.21.15.7:0/3284232300 learned_addr learned my addr 172.21.15.7:0/3284232300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c42000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c10a060 0x7f6c3c1a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 -- 172.21.15.7:0/3284232300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 msgr2=0x7f6c3c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 -- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c10a060 msgr2=0x7f6c3c1a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c10a060 0x7f6c3c1a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.103+0000 7f6c40c00640 1 -- 172.21.15.7:0/3284232300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3c1a6400 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c40c00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e9f0 tx=0x7f6c2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800cd80 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3c1a66f0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28010910 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c280188f0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.104+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3c1a6c40 con 0x7f6c3c1096a0 2024-04-20T21:30:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.105+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c3c10f7d0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.107+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f6c28010ac0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.109+0000 7f6c3a000640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 0x7f6c240a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.109+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6c280d0a80 con 0x7f6c3c1096a0 2024-04-20T21:30:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.109+0000 7f6c41600640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 0x7f6c240a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.110+0000 7f6c41600640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 0x7f6c240a6060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c3002c190 tx=0x7f6c30000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.111+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6c2809cec0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.321+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c3c02d820 con 0x7f6c3c1096a0 2024-04-20T21:30:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.323+0000 7f6c3a000640 1 -- 172.21.15.7:0/3284232300 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22103 (secure 0 0 0) 0x7f6c280a11c0 con 0x7f6c3c1096a0 2024-04-20T21:30:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:54.324 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:30:53.904734+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 msgr2=0x7f6c240a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 0x7f6c240a6060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c3002c190 tx=0x7f6c30000bc0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 msgr2=0x7f6c3c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800e9f0 tx=0x7f6c2800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 shutdown_connections 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6c240a3ba0 0x7f6c240a6060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c3c1096a0 0x7f6c3c19ed20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c3c10a060 0x7f6c3c1a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 --2- 172.21.15.7:0/3284232300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6c3c108500 0x7f6c3c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 >> 172.21.15.7:0/3284232300 conn(0x7f6c3c077450 msgr2=0x7f6c3c10c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 shutdown_connections 2024-04-20T21:30:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.326+0000 7f6c42a00640 1 -- 172.21.15.7:0/3284232300 wait complete. 2024-04-20T21:30:54.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 -- 172.21.15.7:0/2196366533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4074960 msgr2=0x7fbdb4074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/2196366533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4074960 0x7fbdb4074dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009900 tx=0x7fbd9c02bcc0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 -- 172.21.15.7:0/2196366533 shutdown_connections 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/2196366533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb410c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/2196366533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4074960 0x7fbdb4074dc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/2196366533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4075f40 0x7fbdb4074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.550+0000 7fbdb3e00640 1 -- 172.21.15.7:0/2196366533 >> 172.21.15.7:0/2196366533 conn(0x7fbdb40fd420 msgr2=0x7fbdb40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.551+0000 7fbdb3e00640 1 -- 172.21.15.7:0/2196366533 shutdown_connections 2024-04-20T21:30:54.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.551+0000 7fbdb3e00640 1 -- 172.21.15.7:0/2196366533 wait complete. 2024-04-20T21:30:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 Processor -- start 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 -- start start 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4075f40 0x7fbdb419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb41a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410fd90 con 0x7fbdb4074960 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410fc10 con 0x7fbdb4109d40 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb410ff10 con 0x7fbdb4075f40 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34002/0 (socket says 172.21.15.7:34002) 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb2a00640 1 -- 172.21.15.7:0/1200523071 learned_addr learned my addr 172.21.15.7:0/1200523071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb2000640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4075f40 0x7fbdb419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.552+0000 7fbdb3400640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb41a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 -- 172.21.15.7:0/1200523071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4075f40 msgr2=0x7fbdb419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4075f40 0x7fbdb419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 -- 172.21.15.7:0/1200523071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 msgr2=0x7fbdb41a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb41a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 -- 172.21.15.7:0/1200523071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb41a1fc0 con 0x7fbdb4074960 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb3400640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb2a00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009d50 tx=0x7fbda400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4003b70 con 0x7fbdb4074960 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda4003d10 con 0x7fbdb4074960 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41a22b0 con 0x7fbdb4074960 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.553+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4010a20 con 0x7fbdb4074960 2024-04-20T21:30:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.554+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41a27d0 con 0x7fbdb4074960 2024-04-20T21:30:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.555+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4108b00 con 0x7fbdb4074960 2024-04-20T21:30:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.557+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbda4004000 con 0x7fbdb4074960 2024-04-20T21:30:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.559+0000 7fbdab400640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 0x7fbd980a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.559+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbda40d0a70 con 0x7fbdb4074960 2024-04-20T21:30:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.559+0000 7fbdb2000640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 0x7fbd980a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.560+0000 7fbdb2000640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 0x7fbd980a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c02c2d0 tx=0x7fbd9c02c910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.564+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbda409d500 con 0x7fbdb4074960 2024-04-20T21:30:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.771+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdb402d5d0 con 0x7fbdb4074960 2024-04-20T21:30:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.775+0000 7fbdab400640 1 -- 172.21.15.7:0/1200523071 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22103 (secure 0 0 0) 0x7fbda409ce00 con 0x7fbdb4074960 2024-04-20T21:30:54.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:30:54.776 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:30:53.904734+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 msgr2=0x7fbd980a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 0x7fbd980a6060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c02c2d0 tx=0x7fbd9c02c910 comp rx=0 tx=0).stop 2024-04-20T21:30:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 msgr2=0x7fbdb419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009d50 tx=0x7fbda400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 shutdown_connections 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbd980a3ba0 0x7fbd980a6060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbdb4109d40 0x7fbdb41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbdb4075f40 0x7fbdb419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 --2- 172.21.15.7:0/1200523071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdb4074960 0x7fbdb419a3a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 >> 172.21.15.7:0/1200523071 conn(0x7fbdb40fd420 msgr2=0x7fbdb4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 shutdown_connections 2024-04-20T21:30:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:54.779+0000 7fbdb3e00640 1 -- 172.21.15.7:0/1200523071 wait complete. 2024-04-20T21:30:54.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 -- 172.21.15.7:0/3675458353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 msgr2=0x7fe71c104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 --2- 172.21.15.7:0/3675458353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c104c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009900 tx=0x7fe70c02bc50 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 -- 172.21.15.7:0/3675458353 shutdown_connections 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 --2- 172.21.15.7:0/3675458353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c107ab0 0x7fe71c109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 --2- 172.21.15.7:0/3675458353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c105180 0x7fe71c107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 --2- 172.21.15.7:0/3675458353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c104c40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 -- 172.21.15.7:0/3675458353 >> 172.21.15.7:0/3675458353 conn(0x7fe71c0fc240 msgr2=0x7fe71c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 -- 172.21.15.7:0/3675458353 shutdown_connections 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.020+0000 7fe721800640 1 -- 172.21.15.7:0/3675458353 wait complete. 2024-04-20T21:30:55.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.021+0000 7fe721800640 1 Processor -- start 2024-04-20T21:30:55.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.021+0000 7fe721800640 1 -- start start 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c105180 0x7fe71c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c107ab0 0x7fe71c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10daa0 con 0x7fe71c102850 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10d920 con 0x7fe71c105180 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe721800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10dc20 con 0x7fe71c107ab0 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe71be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe71be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34028/0 (socket says 172.21.15.7:34028) 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe71be00640 1 -- 172.21.15.7:0/476571311 learned_addr learned my addr 172.21.15.7:0/476571311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe71b400640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c105180 0x7fe71c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe720e00640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c107ab0 0x7fe71c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.022+0000 7fe71be00640 1 -- 172.21.15.7:0/476571311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c107ab0 msgr2=0x7fe71c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe71be00640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c107ab0 0x7fe71c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe71be00640 1 -- 172.21.15.7:0/476571311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c105180 msgr2=0x7fe71c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe71be00640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c105180 0x7fe71c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe71be00640 1 -- 172.21.15.7:0/476571311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19fd70 con 0x7fe71c102850 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe71be00640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02c260 tx=0x7fe70c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c041070 con 0x7fe71c102850 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1a0000 con 0x7fe71c102850 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c0054e0 con 0x7fe71c102850 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0346a0 con 0x7fe71c102850 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.023+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a05a0 con 0x7fe71c102850 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.025+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c0fdd30 con 0x7fe71c102850 2024-04-20T21:30:55.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.027+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fe70c044070 con 0x7fe71c102850 2024-04-20T21:30:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.028+0000 7fe718c00640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 0x7fe7000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.028+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe70c0f1f70 con 0x7fe71c102850 2024-04-20T21:30:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.028+0000 7fe71b400640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 0x7fe7000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.029+0000 7fe71b400640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 0x7fe7000a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe71c199660 tx=0x7fe70400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.034+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe70c003e20 con 0x7fe71c102850 2024-04-20T21:30:55.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:30:55.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:30:55.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:30:55.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:30:55.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3284232300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.196 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:54 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1200523071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:30:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:30:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:30:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:30:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3284232300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:54 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1200523071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4173939274' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: osdmap e86: 12 total, 12 up, 12 in 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3284232300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:54 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1200523071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:30:55.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.289+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe71c02d5d0 con 0x7fe71c102850 2024-04-20T21:30:55.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.911+0000 7fe718c00640 1 -- 172.21.15.7:0/476571311 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fe70c0be3b0 con 0x7fe71c102850 2024-04-20T21:30:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.914+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 msgr2=0x7fe7000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.914+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 0x7fe7000a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe71c199660 tx=0x7fe70400b410 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.914+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 msgr2=0x7fe71c198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.914+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02c260 tx=0x7fe70c005dd0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 shutdown_connections 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7000a3bb0 0x7fe7000a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe71c105180 0x7fe71c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe71c107ab0 0x7fe71c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 --2- 172.21.15.7:0/476571311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c102850 0x7fe71c198150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 >> 172.21.15.7:0/476571311 conn(0x7fe71c0fc240 msgr2=0x7fe71c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 shutdown_connections 2024-04-20T21:30:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:55.915+0000 7fe721800640 1 -- 172.21.15.7:0/476571311 wait complete. 2024-04-20T21:30:55.939 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-20T21:30:55.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-20T21:30:56.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.157+0000 7ffa26600640 1 -- 172.21.15.7:0/4251420178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa20105bb0 msgr2=0x7ffa2010c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.157+0000 7ffa26600640 1 --2- 172.21.15.7:0/4251420178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa20105bb0 0x7ffa2010c440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009be0 tx=0x7ffa1802bef0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 -- 172.21.15.7:0/4251420178 shutdown_connections 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 --2- 172.21.15.7:0/4251420178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa201051f0 0x7ffa20105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 --2- 172.21.15.7:0/4251420178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa20105bb0 0x7ffa2010c440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 --2- 172.21.15.7:0/4251420178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20104000 0x7ffa20104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 -- 172.21.15.7:0/4251420178 >> 172.21.15.7:0/4251420178 conn(0x7ffa200ff570 msgr2=0x7ffa201019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 -- 172.21.15.7:0/4251420178 shutdown_connections 2024-04-20T21:30:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.158+0000 7ffa26600640 1 -- 172.21.15.7:0/4251420178 wait complete. 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 Processor -- start 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 -- start start 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 0x7ffa2019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa201051f0 0x7ffa2019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 0x7ffa201a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010fca0 con 0x7ffa20105bb0 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010fb20 con 0x7ffa20104000 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.159+0000 7ffa26600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2010fe20 con 0x7ffa201051f0 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 0x7ffa201a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 0x7ffa2019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 0x7ffa201a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34040/0 (socket says 172.21.15.7:34040) 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25c00640 1 -- 172.21.15.7:0/155686058 learned_addr learned my addr 172.21.15.7:0/155686058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa1fe00640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa201051f0 0x7ffa2019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 -- 172.21.15.7:0/155686058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa201051f0 msgr2=0x7ffa2019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa201051f0 0x7ffa2019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 -- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 msgr2=0x7ffa201a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 0x7ffa201a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 -- 172.21.15.7:0/155686058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201a1eb0 con 0x7ffa20104000 2024-04-20T21:30:56.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.160+0000 7ffa25200640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 0x7ffa2019a290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005fd0 tx=0x7ffa1000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.161+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000ccc0 con 0x7ffa20104000 2024-04-20T21:30:56.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.161+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201a21a0 con 0x7ffa20104000 2024-04-20T21:30:56.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.161+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1000ee70 con 0x7ffa20104000 2024-04-20T21:30:56.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.161+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10010430 con 0x7ffa20104000 2024-04-20T21:30:56.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.161+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a26f0 con 0x7ffa20104000 2024-04-20T21:30:56.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.162+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2010b2c0 con 0x7ffa20104000 2024-04-20T21:30:56.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.164+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ffa10003ef0 con 0x7ffa20104000 2024-04-20T21:30:56.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.165+0000 7ffa1d600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 0x7ffa040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.166+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffa100d0680 con 0x7ffa20104000 2024-04-20T21:30:56.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.166+0000 7ffa1fe00640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 0x7ffa040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.166+0000 7ffa1fe00640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 0x7ffa040a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa2019b7a0 tx=0x7ffa08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.170+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffa1009d110 con 0x7ffa20104000 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:active 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:30:56.197 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:55 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:30:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:active 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:30:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:55 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: pgmap v319: 97 pgs: 17 creating+peering, 40 active+clean, 40 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] up:active 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:30:56.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:55 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-20T21:30:56.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.420+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ffa20062b60 con 0x7ffa20104000 2024-04-20T21:30:56.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.917+0000 7ffa1d600640 1 -- 172.21.15.7:0/155686058 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ffa1009ca10 con 0x7ffa20104000 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.920+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 msgr2=0x7ffa040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.920+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 0x7ffa040a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa2019b7a0 tx=0x7ffa08005f90 comp rx=0 tx=0).stop 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 msgr2=0x7ffa2019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 0x7ffa2019a290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005fd0 tx=0x7ffa1000ec20 comp rx=0 tx=0).stop 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 shutdown_connections 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa040a3bb0 0x7ffa040a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa20105bb0 0x7ffa201a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa201051f0 0x7ffa2019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 --2- 172.21.15.7:0/155686058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20104000 0x7ffa2019a290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 >> 172.21.15.7:0/155686058 conn(0x7ffa200ff570 msgr2=0x7ffa20108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 shutdown_connections 2024-04-20T21:30:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:56.921+0000 7ffa26600640 1 -- 172.21.15.7:0/155686058 wait complete. 2024-04-20T21:30:56.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-20T21:30:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 -- 172.21.15.7:0/3389831795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 msgr2=0x7f4764102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 --2- 172.21.15.7:0/3389831795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f4764102200 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 -- 172.21.15.7:0/3389831795 shutdown_connections 2024-04-20T21:30:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 --2- 172.21.15.7:0/3389831795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f476410a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 --2- 172.21.15.7:0/3389831795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f4764103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 --2- 172.21.15.7:0/3389831795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f4764102200 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 -- 172.21.15.7:0/3389831795 >> 172.21.15.7:0/3389831795 conn(0x7f47640fd370 msgr2=0x7f47640ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 -- 172.21.15.7:0/3389831795 shutdown_connections 2024-04-20T21:30:57.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.158+0000 7f4768a00640 1 -- 172.21.15.7:0/3389831795 wait complete. 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.159+0000 7f4768a00640 1 Processor -- start 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.159+0000 7f4768a00640 1 -- start start 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.159+0000 7f4768a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f476419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.159+0000 7f4768a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4768a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f47641a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4768a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f476410da30 con 0x7f4764102fd0 2024-04-20T21:30:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4768a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f476410d8b0 con 0x7f4764101de0 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4768a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f476410dbb0 con 0x7f4764103990 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f476419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f475aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f475aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34058/0 (socket says 172.21.15.7:34058) 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f475aa00640 1 -- 172.21.15.7:0/3490780923 learned_addr learned my addr 172.21.15.7:0/3490780923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f47641a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 -- 172.21.15.7:0/3490780923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 msgr2=0x7f476419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f476419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 -- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 msgr2=0x7f476419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763e00640 1 -- 172.21.15.7:0/3490780923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641a1f10 con 0x7f4764103990 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f475aa00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.160+0000 7f4763400640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f476419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4763e00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f47641a18a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f475c009fd0 tx=0x7f475c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00ce20 con 0x7f4764103990 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641a2200 con 0x7f4764103990 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c005020 con 0x7f4764103990 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c010720 con 0x7f4764103990 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.161+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641a2700 con 0x7f4764103990 2024-04-20T21:30:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.162+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47641a29c0 con 0x7f4764103990 2024-04-20T21:30:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.164+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f475c005540 con 0x7f4764103990 2024-04-20T21:30:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.166+0000 7f4760c00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 0x7f47480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.166+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f475c0d0a00 con 0x7f4764103990 2024-04-20T21:30:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.166+0000 7f4763400640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 0x7f47480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.167+0000 7f4763400640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 0x7f47480a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4754000c00 tx=0x7f4754005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.170+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f475c09d490 con 0x7f4764103990 2024-04-20T21:30:57.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:56 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:30:57.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:56 smithi016 ceph-mon[21409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:57.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:56 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/155686058' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:56 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:56 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:30:57.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:56 smithi186 ceph-mon[22157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:57.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:56 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/155686058' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:56 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/476571311' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-20T21:30:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:56 smithi007 ceph-mon[16867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/155686058' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:56 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-20T21:30:57.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.421+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f47641a2db0 con 0x7f4764103990 2024-04-20T21:30:57.926 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: 2024-04-20T21:30:57.634+0000 7f4ecac8a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:30:57.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.941+0000 7f4760c00640 1 -- 172.21.15.7:0/3490780923 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f475c09cd90 con 0x7f4764103990 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 msgr2=0x7f47480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 0x7f47480a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4754000c00 tx=0x7f4754005fd0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 msgr2=0x7f47641a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f47641a18a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f475c009fd0 tx=0x7f475c00ef10 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 shutdown_connections 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f47480a3bb0 0x7f47480a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4764103990 0x7f47641a18a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4764102fd0 0x7f476419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 --2- 172.21.15.7:0/3490780923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4764101de0 0x7f476419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 >> 172.21.15.7:0/3490780923 conn(0x7f47640fd370 msgr2=0x7f4764074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 shutdown_connections 2024-04-20T21:30:57.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:57.945+0000 7f4768a00640 1 -- 172.21.15.7:0/3490780923 wait complete. 2024-04-20T21:30:57.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-20T21:30:58.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.171+0000 7ff26d200640 1 -- 172.21.15.7:0/2007281078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 msgr2=0x7ff2680743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.171+0000 7ff26d200640 1 --2- 172.21.15.7:0/2007281078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff2680743f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff258009900 tx=0x7ff25802bc40 comp rx=0 tx=0).stop 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 -- 172.21.15.7:0/2007281078 shutdown_connections 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 --2- 172.21.15.7:0/2007281078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff268109e60 0x7ff26810c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 --2- 172.21.15.7:0/2007281078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2680749c0 0x7ff268074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 --2- 172.21.15.7:0/2007281078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff2680743f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 -- 172.21.15.7:0/2007281078 >> 172.21.15.7:0/2007281078 conn(0x7ff2680fd4f0 msgr2=0x7ff2680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 -- 172.21.15.7:0/2007281078 shutdown_connections 2024-04-20T21:30:58.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.172+0000 7ff26d200640 1 -- 172.21.15.7:0/2007281078 wait complete. 2024-04-20T21:30:58.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 Processor -- start 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 -- start start 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2680749c0 0x7ff26819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 0x7ff2681a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810fe40 con 0x7ff268109e60 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810fcc0 con 0x7ff268075fa0 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff26d200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff26810ffc0 con 0x7ff2680749c0 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.173+0000 7ff25fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45626/0 (socket says 172.21.15.7:45626) 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 -- 172.21.15.7:0/3531048106 learned_addr learned my addr 172.21.15.7:0/3531048106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:58.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff265e00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2680749c0 0x7ff26819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff266800640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 0x7ff2681a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 -- 172.21.15.7:0/3531048106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2680749c0 msgr2=0x7ff26819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2680749c0 0x7ff26819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 -- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 msgr2=0x7ff2681a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 0x7ff2681a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 -- 172.21.15.7:0/3531048106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2681a20d0 con 0x7ff268075fa0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff266800640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 0x7ff2681a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25fe00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff25000d950 tx=0x7ff25000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250017030 con 0x7ff268075fa0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.174+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff250004020 con 0x7ff268075fa0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.175+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2681a23c0 con 0x7ff268075fa0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.175+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2500047a0 con 0x7ff268075fa0 2024-04-20T21:30:58.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.175+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2681a2960 con 0x7ff268075fa0 2024-04-20T21:30:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.176+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff268108b90 con 0x7ff268075fa0 2024-04-20T21:30:58.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.178+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7ff2500041d0 con 0x7ff268075fa0 2024-04-20T21:30:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.179+0000 7ff25ea00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 0x7ff24c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.179+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff2500cfd60 con 0x7ff268075fa0 2024-04-20T21:30:58.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.180+0000 7ff265e00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 0x7ff24c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.180+0000 7ff265e00640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 0x7ff24c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff25802c250 tx=0x7ff25802cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.183+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff25009c7f0 con 0x7ff268075fa0 2024-04-20T21:30:58.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: pgmap v320: 97 pgs: 17 creating+peering, 78 active+clean, 2 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 820 B/s wr, 5 op/s 2024-04-20T21:30:58.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T21:30:58.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3490780923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:57 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: pgmap v320: 97 pgs: 17 creating+peering, 78 active+clean, 2 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 820 B/s wr, 5 op/s 2024-04-20T21:30:58.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3490780923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:57 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: pgmap v320: 97 pgs: 17 creating+peering, 78 active+clean, 2 unknown; 578 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 820 B/s wr, 5 op/s 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3490780923' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:57 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-20T21:30:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:58.434+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff26802d5d0 con 0x7ff268075fa0 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:active 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:active 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3531048106' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.195 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:58 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:active 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:active 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3531048106' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:58 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] up:active 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] up:active 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:30:59.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3531048106' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:58 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-20T21:30:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.406+0000 7ff25ea00640 1 -- 172.21.15.7:0/3531048106 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7ff25009c0f0 con 0x7ff268075fa0 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 msgr2=0x7ff24c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 0x7ff24c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff25802c250 tx=0x7ff25802cf90 comp rx=0 tx=0).stop 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 msgr2=0x7ff26819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff25000d950 tx=0x7ff25000de10 comp rx=0 tx=0).stop 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 shutdown_connections 2024-04-20T21:30:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff24c0a3c30 0x7ff24c0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff268075fa0 0x7ff26819a9f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff268109e60 0x7ff2681a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 --2- 172.21.15.7:0/3531048106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2680749c0 0x7ff26819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.411+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 >> 172.21.15.7:0/3531048106 conn(0x7ff2680fd4f0 msgr2=0x7ff268105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.412+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 shutdown_connections 2024-04-20T21:30:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.412+0000 7ff26d200640 1 -- 172.21.15.7:0/3531048106 wait complete. 2024-04-20T21:30:59.437 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-20T21:30:59.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 -- 172.21.15.7:0/3230865420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 msgr2=0x7fbcb8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 --2- 172.21.15.7:0/3230865420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb8109c30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009900 tx=0x7fbca002bc30 comp rx=0 tx=0).stop 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 -- 172.21.15.7:0/3230865420 shutdown_connections 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 --2- 172.21.15.7:0/3230865420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb8109c30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 --2- 172.21.15.7:0/3230865420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb810a170 0x7fbcb8110a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 --2- 172.21.15.7:0/3230865420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb81085c0 0x7fbcb81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 -- 172.21.15.7:0/3230865420 >> 172.21.15.7:0/3230865420 conn(0x7fbcb8077570 msgr2=0x7fbcb8079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 -- 172.21.15.7:0/3230865420 shutdown_connections 2024-04-20T21:30:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.659+0000 7fbcbf200640 1 -- 172.21.15.7:0/3230865420 wait complete. 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 Processor -- start 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 -- start start 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb81085c0 0x7fbcb819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb810a170 0x7fbcb81a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb81142a0 con 0x7fbcb810a170 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8114120 con 0x7fbcb81097b0 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcbf200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8114420 con 0x7fbcb81085c0 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.660+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45648/0 (socket says 172.21.15.7:45648) 2024-04-20T21:30:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 -- 172.21.15.7:0/2010550329 learned_addr learned my addr 172.21.15.7:0/2010550329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcbd200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb810a170 0x7fbcb81a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcb7e00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb81085c0 0x7fbcb819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 -- 172.21.15.7:0/2010550329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb81085c0 msgr2=0x7fbcb819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb81085c0 0x7fbcb819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 -- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb810a170 msgr2=0x7fbcb81a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb810a170 0x7fbcb81a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 -- 172.21.15.7:0/2010550329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb81a6500 con 0x7fbcb81097b0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcafe00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbca0005ab0 tx=0x7fbca002c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.661+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0043020 con 0x7fbcb81097b0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.662+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca00054e0 con 0x7fbcb81097b0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.662+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0003ce0 con 0x7fbcb81097b0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.662+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81a6790 con 0x7fbcb81097b0 2024-04-20T21:30:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.662+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a6c90 con 0x7fbcb81097b0 2024-04-20T21:30:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.663+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb810f960 con 0x7fbcb81097b0 2024-04-20T21:30:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.665+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7fbca0046020 con 0x7fbcb81097b0 2024-04-20T21:30:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.666+0000 7fbcb5600640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 0x7fbc9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.666+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbca00f2390 con 0x7fbcb81097b0 2024-04-20T21:30:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.666+0000 7fbcb7e00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 0x7fbc9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.667+0000 7fbcb7e00640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 0x7fbc9c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8109610 tx=0x7fbca800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.674+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbca004d020 con 0x7fbcb81097b0 2024-04-20T21:30:59.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.911+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fbcb80623a0 con 0x7fbc9c0a3c30 2024-04-20T21:30:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.946+0000 7fbcb5600640 1 -- 172.21.15.7:0/2010550329 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbcb80623a0 con 0x7fbc9c0a3c30 2024-04-20T21:30:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 msgr2=0x7fbc9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 0x7fbc9c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8109610 tx=0x7fbca800b390 comp rx=0 tx=0).stop 2024-04-20T21:30:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 msgr2=0x7fbcb819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbca0005ab0 tx=0x7fbca002c6c0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 shutdown_connections 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbc9c0a3c30 0x7fbc9c0a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb81097b0 0x7fbcb819ee20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb810a170 0x7fbcb81a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 --2- 172.21.15.7:0/2010550329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcb81085c0 0x7fbcb819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.949+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 >> 172.21.15.7:0/2010550329 conn(0x7fbcb8077570 msgr2=0x7fbcb810cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.950+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 shutdown_connections 2024-04-20T21:30:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:30:59.950+0000 7fbcbf200640 1 -- 172.21.15.7:0/2010550329 wait complete. 2024-04-20T21:30:59.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 -- 172.21.15.7:0/3198137046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 msgr2=0x7f30d8108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 --2- 172.21.15.7:0/3198137046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d8108800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009900 tx=0x7f30c802bc50 comp rx=0 tx=0).stop 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 -- 172.21.15.7:0/3198137046 shutdown_connections 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 --2- 172.21.15.7:0/3198137046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d8109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 --2- 172.21.15.7:0/3198137046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 0x7f30d8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 --2- 172.21.15.7:0/3198137046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d8108800 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 -- 172.21.15.7:0/3198137046 >> 172.21.15.7:0/3198137046 conn(0x7f30d80773a0 msgr2=0x7f30d80797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 -- 172.21.15.7:0/3198137046 shutdown_connections 2024-04-20T21:31:00.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.179+0000 7f30d7200640 1 -- 172.21.15.7:0/3198137046 wait complete. 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 Processor -- start 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 -- start start 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 0x7f30d81a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8114100 con 0x7f30d81083e0 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8113f80 con 0x7f30d81095d0 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d7200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30d8114280 con 0x7f30d8109f90 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.180+0000 7f30d5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34164/0 (socket says 172.21.15.7:34164) 2024-04-20T21:31:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 -- 172.21.15.7:0/1048460136 learned_addr learned my addr 172.21.15.7:0/1048460136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d6800640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 0x7f30d81a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5400640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 -- 172.21.15.7:0/1048460136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 msgr2=0x7f30d81a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 0x7f30d81a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 -- 172.21.15.7:0/1048460136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 msgr2=0x7f30d819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 -- 172.21.15.7:0/1048460136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d81a63c0 con 0x7f30d81083e0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5400640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.181+0000 7f30d5e00640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005ab0 tx=0x7f30c802c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.182+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8041070 con 0x7f30d81083e0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.182+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8004450 con 0x7f30d81083e0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.182+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8034440 con 0x7f30d81083e0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.182+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d81a6650 con 0x7f30d81083e0 2024-04-20T21:31:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.182+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d81a6bf0 con 0x7f30d81083e0 2024-04-20T21:31:00.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.183+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d810f700 con 0x7f30d81083e0 2024-04-20T21:31:00.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.187+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f30c8044070 con 0x7f30d81083e0 2024-04-20T21:31:00.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.188+0000 7f30cea00640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 0x7f30bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.188+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f30c80f2300 con 0x7f30d81083e0 2024-04-20T21:31:00.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.189+0000 7f30d5400640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 0x7f30bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.189+0000 7f30d5400640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 0x7f30bc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30d819fcb0 tx=0x7f30c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.191+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f30c80bed90 con 0x7f30d81083e0 2024-04-20T21:31:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:59 smithi186 ceph-mon[22157]: pgmap v321: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-20T21:31:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:59 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:00.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:30:59 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-mon[16867]: pgmap v321: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:30:59.922+0000 7f5135400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:30:59.922+0000 7f5135400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:30:59.922+0000 7f5135400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:00.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:30:59.922+0000 7f5135400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:00.284 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:30:59 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:30:59.922+0000 7f5135400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:31:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.435+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f30d80623a0 con 0x7f30bc0a3c30 2024-04-20T21:31:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:59 smithi016 ceph-mon[21409]: pgmap v321: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-20T21:31:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:59 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-20T21:31:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:30:59 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:31:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.469+0000 7f30cea00640 1 -- 172.21.15.7:0/1048460136 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30d80623a0 con 0x7f30bc0a3c30 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 msgr2=0x7f30bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 0x7f30bc0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30d819fcb0 tx=0x7f30c4005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 msgr2=0x7f30d819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005ab0 tx=0x7f30c802c6e0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 shutdown_connections 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f30bc0a3c30 0x7f30bc0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d81095d0 0x7f30d819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30d8109f90 0x7f30d81a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 --2- 172.21.15.7:0/1048460136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d81083e0 0x7f30d819e7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 >> 172.21.15.7:0/1048460136 conn(0x7f30d80773a0 msgr2=0x7f30d810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 shutdown_connections 2024-04-20T21:31:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.472+0000 7f30d7200640 1 -- 172.21.15.7:0/1048460136 wait complete. 2024-04-20T21:31:00.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 -- 172.21.15.7:0/3662712527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc076920 msgr2=0x7f96dc076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 --2- 172.21.15.7:0/3662712527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc076920 0x7f96dc076d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009900 tx=0x7f96d002bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 -- 172.21.15.7:0/3662712527 shutdown_connections 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 --2- 172.21.15.7:0/3662712527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc077b60 0x7f96dc077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 --2- 172.21.15.7:0/3662712527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 --2- 172.21.15.7:0/3662712527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc076920 0x7f96dc076d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 -- 172.21.15.7:0/3662712527 >> 172.21.15.7:0/3662712527 conn(0x7f96dc0ff570 msgr2=0x7f96dc101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 -- 172.21.15.7:0/3662712527 shutdown_connections 2024-04-20T21:31:00.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.713+0000 7f96e2a00640 1 -- 172.21.15.7:0/3662712527 wait complete. 2024-04-20T21:31:00.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.714+0000 7f96e2a00640 1 Processor -- start 2024-04-20T21:31:00.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.714+0000 7f96e2a00640 1 -- start start 2024-04-20T21:31:00.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 0x7f96dc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc077b60 0x7f96dc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc114070 con 0x7f96dc078500 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc113ef0 con 0x7f96dc076920 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1141f0 con 0x7f96dc077b60 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34188/0 (socket says 172.21.15.7:34188) 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e2000640 1 -- 172.21.15.7:0/9353318 learned_addr learned my addr 172.21.15.7:0/9353318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e1600640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 0x7f96dc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.715+0000 7f96e0c00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc077b60 0x7f96dc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 -- 172.21.15.7:0/9353318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc077b60 msgr2=0x7f96dc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc077b60 0x7f96dc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 -- 172.21.15.7:0/9353318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 msgr2=0x7f96dc19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 0x7f96dc19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 -- 172.21.15.7:0/9353318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1a6320 con 0x7f96dc078500 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e1600640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 0x7f96dc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2000640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96d400efd0 tx=0x7f96d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003b70 con 0x7f96dc078500 2024-04-20T21:31:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d4003d10 con 0x7f96dc078500 2024-04-20T21:31:00.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1a6610 con 0x7f96dc078500 2024-04-20T21:31:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d40109f0 con 0x7f96dc078500 2024-04-20T21:31:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.716+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1a6b10 con 0x7f96dc078500 2024-04-20T21:31:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.718+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc110950 con 0x7f96dc078500 2024-04-20T21:31:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.720+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 16) ==== 138645+0+0 (secure 0 0 0) 0x7f96d4004020 con 0x7f96dc078500 2024-04-20T21:31:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.721+0000 7f96da000640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 0x7f96c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.721+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96d40d07a0 con 0x7f96dc078500 2024-04-20T21:31:00.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.722+0000 7f96e1600640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 0x7f96c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.722+0000 7f96e1600640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 0x7f96c40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96d002c7b0 tx=0x7f96d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.726+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f96d409d230 con 0x7f96dc078500 2024-04-20T21:31:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.982+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f96dc0623a0 con 0x7f96c40a3bb0 2024-04-20T21:31:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.987+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f96d4004310 con 0x7f96dc078500 2024-04-20T21:31:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:00.999+0000 7f96da000640 1 -- 172.21.15.7:0/9353318 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f96dc0623a0 con 0x7f96c40a3bb0 2024-04-20T21:31:00.999 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_0/d341b50f-d6f0-42ed-b7a4-47f5ff842080 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 msgr2=0x7f96c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 0x7f96c40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96d002c7b0 tx=0x7f96d0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 msgr2=0x7f96dc1a5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f96d400efd0 tx=0x7f96d400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 shutdown_connections 2024-04-20T21:31:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f96c40a3bb0 0x7f96c40a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96dc078500 0x7f96dc1a5cb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96dc077b60 0x7f96dc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 --2- 172.21.15.7:0/9353318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96dc076920 0x7f96dc19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.002+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 >> 172.21.15.7:0/9353318 conn(0x7f96dc0ff570 msgr2=0x7f96dc1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.003+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 shutdown_connections 2024-04-20T21:31:01.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.003+0000 7f96e2a00640 1 -- 172.21.15.7:0/9353318 wait complete. 2024-04-20T21:31:01.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 -- 172.21.15.7:0/982788253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 msgr2=0x7f25a8077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 --2- 172.21.15.7:0/982788253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a8077fc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2594009900 tx=0x7f259402bc40 comp rx=0 tx=0).stop 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 -- 172.21.15.7:0/982788253 shutdown_connections 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 --2- 172.21.15.7:0/982788253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a8078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 --2- 172.21.15.7:0/982788253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a8077fc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 --2- 172.21.15.7:0/982788253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a8076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 -- 172.21.15.7:0/982788253 >> 172.21.15.7:0/982788253 conn(0x7f25a80ff570 msgr2=0x7f25a8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 -- 172.21.15.7:0/982788253 shutdown_connections 2024-04-20T21:31:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.234+0000 7f25ae400640 1 -- 172.21.15.7:0/982788253 wait complete. 2024-04-20T21:31:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.235+0000 7f25ae400640 1 Processor -- start 2024-04-20T21:31:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.235+0000 7f25ae400640 1 -- start start 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a81a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25a81140b0 con 0x7f25a8077b60 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8113f30 con 0x7f25a8076920 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ae400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8114230 con 0x7f25a8078500 2024-04-20T21:31:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34208/0 (socket says 172.21.15.7:34208) 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25a7e00640 1 -- 172.21.15.7:0/2102567581 learned_addr learned my addr 172.21.15.7:0/2102567581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.236+0000 7f25ada00640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a81a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 -- 172.21.15.7:0/2102567581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 msgr2=0x7f25a81a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a81a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 -- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 msgr2=0x7f25a819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 -- 172.21.15.7:0/2102567581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a81a6330 con 0x7f25a8076920 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ada00640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a81a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25a7e00640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ad000640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a819e710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e9f0 tx=0x7f259c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c00cd80 con 0x7f25a8076920 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f259c0035f0 con 0x7f25a8076920 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a81a6620 con 0x7f25a8076920 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c010670 con 0x7f25a8076920 2024-04-20T21:31:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.237+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a81a6b40 con 0x7f25a8076920 2024-04-20T21:31:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.239+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a8110950 con 0x7f25a8076920 2024-04-20T21:31:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.240+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f259c0037a0 con 0x7f25a8076920 2024-04-20T21:31:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.242+0000 7f25a5600640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 0x7f25900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.242+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f259c0d07b0 con 0x7f25a8076920 2024-04-20T21:31:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.242+0000 7f25a7e00640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 0x7f25900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.243+0000 7f25a7e00640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 0x7f25900a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2594006010 tx=0x7f259402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.246+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f259c09d0e0 con 0x7f25a8076920 2024-04-20T21:31:01.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:00 smithi007 ceph-mon[16867]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:01.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:00 smithi016 ceph-mon[21409]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:01.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:00 smithi186 ceph-mon[22157]: from='client.24659 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:01.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.482+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f25a80623a0 con 0x7f25900a3c40 2024-04-20T21:31:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.517+0000 7f25a5600640 1 -- 172.21.15.7:0/2102567581 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25a80623a0 con 0x7f25900a3c40 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 msgr2=0x7f25900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 0x7f25900a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2594006010 tx=0x7f259402cf90 comp rx=0 tx=0).stop 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 msgr2=0x7f25a819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a819e710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e9f0 tx=0x7f259c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 shutdown_connections 2024-04-20T21:31:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f25900a3c40 0x7f25900a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25a8078500 0x7f25a81a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25a8077b60 0x7f25a819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 --2- 172.21.15.7:0/2102567581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a8076920 0x7f25a819e710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 >> 172.21.15.7:0/2102567581 conn(0x7f25a80ff570 msgr2=0x7f25a8101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.520+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 shutdown_connections 2024-04-20T21:31:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.521+0000 7f25ae400640 1 -- 172.21.15.7:0/2102567581 wait complete. 2024-04-20T21:31:01.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-20T21:31:01.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.756+0000 7f84a3e00640 1 -- 172.21.15.7:0/1845899031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4100640 msgr2=0x7f84a4102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.756+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1845899031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4100640 0x7f84a4102a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8494009be0 tx=0x7f849402bef0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 -- 172.21.15.7:0/1845899031 shutdown_connections 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1845899031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 0x7f84a4105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1845899031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4102fb0 0x7f84a4105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1845899031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4100640 0x7f84a4102a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 -- 172.21.15.7:0/1845899031 >> 172.21.15.7:0/1845899031 conn(0x7f84a40fc240 msgr2=0x7f84a40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 -- 172.21.15.7:0/1845899031 shutdown_connections 2024-04-20T21:31:01.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.757+0000 7f84a3e00640 1 -- 172.21.15.7:0/1845899031 wait complete. 2024-04-20T21:31:01.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.758+0000 7f84a3e00640 1 Processor -- start 2024-04-20T21:31:01.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.758+0000 7f84a3e00640 1 -- start start 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4100640 0x7f84a419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 0x7f84a41a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84a41140c0 con 0x7f84a4105960 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4113f40 con 0x7f84a4102fb0 2024-04-20T21:31:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4114240 con 0x7f84a4100640 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45734/0 (socket says 172.21.15.7:45734) 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4100640 0x7f84a419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 -- 172.21.15.7:0/1603302881 learned_addr learned my addr 172.21.15.7:0/1603302881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a3400640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 0x7f84a41a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 -- 172.21.15.7:0/1603302881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4100640 msgr2=0x7f84a419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4100640 0x7f84a419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 -- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 msgr2=0x7f84a41a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 0x7f84a41a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.759+0000 7f84a2000640 1 -- 172.21.15.7:0/1603302881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a41a62f0 con 0x7f84a4102fb0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f84a2000640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849000cd80 con 0x7f84a4102fb0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8490005500 con 0x7f84a4102fb0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a41a65e0 con 0x7f84a4102fb0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490010690 con 0x7f84a4102fb0 2024-04-20T21:31:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.760+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a6b30 con 0x7f84a4102fb0 2024-04-20T21:31:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.762+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a40fdd30 con 0x7f84a4102fb0 2024-04-20T21:31:01.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.764+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f84900046c0 con 0x7f84a4102fb0 2024-04-20T21:31:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.766+0000 7f849b400640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 0x7f84880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.766+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f84900d0860 con 0x7f84a4102fb0 2024-04-20T21:31:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.766+0000 7f84a2a00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 0x7f84880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.767+0000 7f84a2a00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 0x7f84880a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:01.770+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f849009d190 con 0x7f84a4102fb0 2024-04-20T21:31:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.017+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f84a40623a0 con 0x7f84880a3cc0 2024-04-20T21:31:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.023+0000 7f849b400640 1 -- 172.21.15.7:0/1603302881 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f84a40623a0 con 0x7f84880a3cc0 2024-04-20T21:31:02.023 INFO:teuthology.orchestra.run.smithi007.stdout:/volumes/qa/sv_1/6d643e23-dc51-46e4-949f-950c298bfaa4 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 msgr2=0x7f84880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 0x7f84880a6180 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8494006010 tx=0x7f8494000f40 comp rx=0 tx=0).stop 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 msgr2=0x7f84a419ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f849000e9f0 tx=0x7f849000edf0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 shutdown_connections 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f84880a3cc0 0x7f84880a6180 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a4102fb0 0x7f84a419ec60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84a4105960 0x7f84a41a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 --2- 172.21.15.7:0/1603302881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f84a4100640 0x7f84a419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 >> 172.21.15.7:0/1603302881 conn(0x7f84a40fc240 msgr2=0x7f84a4103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 shutdown_connections 2024-04-20T21:31:02.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.026+0000 7f84a3e00640 1 -- 172.21.15.7:0/1603302881 wait complete. 2024-04-20T21:31:02.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 -- 172.21.15.7:0/4255067884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 msgr2=0x7f8334109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 --2- 172.21.15.7:0/4255067884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f8334109c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:31:02.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 -- 172.21.15.7:0/4255067884 shutdown_connections 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 --2- 172.21.15.7:0/4255067884 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f8334110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 --2- 172.21.15.7:0/4255067884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f8334109c50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 --2- 172.21.15.7:0/4255067884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 0x7f8334108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 -- 172.21.15.7:0/4255067884 >> 172.21.15.7:0/4255067884 conn(0x7f83340775b0 msgr2=0x7f83340799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 -- 172.21.15.7:0/4255067884 shutdown_connections 2024-04-20T21:31:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.261+0000 7f833b800640 1 -- 172.21.15.7:0/4255067884 wait complete. 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.262+0000 7f833b800640 1 Processor -- start 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 -- start start 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 0x7f833419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f83341a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83341142a0 con 0x7f83341097d0 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8334114120 con 0x7f83341085e0 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f833b800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8334114420 con 0x7f833410a190 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f832be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f832be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34246/0 (socket says 172.21.15.7:34246) 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f832be00640 1 -- 172.21.15.7:0/825265955 learned_addr learned my addr 172.21.15.7:0/825265955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f8339800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f83341a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.263+0000 7f8338e00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 0x7f833419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 -- 172.21.15.7:0/825265955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 msgr2=0x7f83341a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f83341a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 -- 172.21.15.7:0/825265955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 msgr2=0x7f833419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 0x7f833419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 -- 172.21.15.7:0/825265955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83341a6500 con 0x7f83341097d0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f8339800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f83341a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f832be00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f831c02c8f0 tx=0x7f831c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.264+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c041070 con 0x7f83341097d0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.265+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83341a6790 con 0x7f83341097d0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.265+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831c0044d0 con 0x7f83341097d0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.265+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c034440 con 0x7f83341097d0 2024-04-20T21:31:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.265+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a6cd0 con 0x7f83341097d0 2024-04-20T21:31:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.266+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833410f8a0 con 0x7f83341097d0 2024-04-20T21:31:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.268+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f831c044070 con 0x7f83341097d0 2024-04-20T21:31:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.269+0000 7f8332000640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 0x7f83140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.270+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f831c0f1ea0 con 0x7f83341097d0 2024-04-20T21:31:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.270+0000 7f8338e00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 0x7f83140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.270+0000 7f8338e00640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 0x7f83140a6180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8334109630 tx=0x7f8324005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.274+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f831c0be290 con 0x7f83341097d0 2024-04-20T21:31:02.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:01 smithi007 ceph-mon[16867]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 512 B/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-20T21:31:02.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:01 smithi007 ceph-mon[16867]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:01 smithi007 ceph-mon[16867]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:01 smithi007 ceph-mon[16867]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-20T21:31:02.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:01 smithi016 ceph-mon[21409]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 512 B/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-20T21:31:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:01 smithi016 ceph-mon[21409]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:01 smithi016 ceph-mon[21409]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:01 smithi016 ceph-mon[21409]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-20T21:31:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:01 smithi186 ceph-mon[22157]: pgmap v322: 97 pgs: 97 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 512 B/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-20T21:31:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:01 smithi186 ceph-mon[22157]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:01 smithi186 ceph-mon[22157]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:01 smithi186 ceph-mon[22157]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-20T21:31:02.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.518+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f833402d5d0 con 0x7f83341097d0 2024-04-20T21:31:02.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.520+0000 7f8332000640 1 -- 172.21.15.7:0/825265955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f831c0c2590 con 0x7f83341097d0 2024-04-20T21:31:02.520 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:02.520 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:31:02.520 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 msgr2=0x7f83140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 0x7f83140a6180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8334109630 tx=0x7f8324005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 msgr2=0x7f833419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f831c02c8f0 tx=0x7f831c005d50 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 shutdown_connections 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f83140a3cc0 0x7f83140a6180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f833410a190 0x7f83341a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83341097d0 0x7f833419ee20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 --2- 172.21.15.7:0/825265955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83341085e0 0x7f833419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 >> 172.21.15.7:0/825265955 conn(0x7f83340775b0 msgr2=0x7f833410ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 shutdown_connections 2024-04-20T21:31:02.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.522+0000 7f833b800640 1 -- 172.21.15.7:0/825265955 wait complete. 2024-04-20T21:31:02.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:02.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.754+0000 7fb83b400640 1 -- 172.21.15.7:0/869840377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 msgr2=0x7fb834074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.754+0000 7fb83b400640 1 --2- 172.21.15.7:0/869840377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb834074dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb820009860 tx=0x7fb82002bc00 comp rx=0 tx=0).stop 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 -- 172.21.15.7:0/869840377 shutdown_connections 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 --2- 172.21.15.7:0/869840377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 0x7fb83410c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 --2- 172.21.15.7:0/869840377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb834074dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 --2- 172.21.15.7:0/869840377 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 0x7fb834074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 -- 172.21.15.7:0/869840377 >> 172.21.15.7:0/869840377 conn(0x7fb8340fd540 msgr2=0x7fb8340ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 -- 172.21.15.7:0/869840377 shutdown_connections 2024-04-20T21:31:02.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.755+0000 7fb83b400640 1 -- 172.21.15.7:0/869840377 wait complete. 2024-04-20T21:31:02.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 Processor -- start 2024-04-20T21:31:02.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 -- start start 2024-04-20T21:31:02.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 0x7fb83419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 0x7fb8341a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410fe60 con 0x7fb834074960 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410fce0 con 0x7fb834109e60 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb83b400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb83410ffe0 con 0x7fb834075f40 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb838a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb838a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34274/0 (socket says 172.21.15.7:34274) 2024-04-20T21:31:02.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.756+0000 7fb838a00640 1 -- 172.21.15.7:0/2545001031 learned_addr learned my addr 172.21.15.7:0/2545001031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb833e00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 0x7fb83419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb839400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 0x7fb8341a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 -- 172.21.15.7:0/2545001031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 msgr2=0x7fb83419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 0x7fb83419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 -- 172.21.15.7:0/2545001031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 msgr2=0x7fb8341a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 0x7fb8341a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 -- 172.21.15.7:0/2545001031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8341a20b0 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.757+0000 7fb838a00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9f0 tx=0x7fb82400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.758+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82400cd80 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.758+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824005500 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.758+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824010690 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.758+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8341a23a0 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.758+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8341a2870 con 0x7fb834074960 2024-04-20T21:31:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.759+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb834108c20 con 0x7fb834074960 2024-04-20T21:31:02.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.761+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb8240046c0 con 0x7fb834074960 2024-04-20T21:31:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.763+0000 7fb831600640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 0x7fb8180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.763+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb8240d0860 con 0x7fb834074960 2024-04-20T21:31:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.763+0000 7fb833e00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 0x7fb8180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.764+0000 7fb833e00640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 0x7fb8180a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb82002c210 tx=0x7fb820034b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.767+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb82409d190 con 0x7fb834074960 2024-04-20T21:31:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.967+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb834062b60 con 0x7fb834074960 2024-04-20T21:31:02.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.971+0000 7fb831600640 1 -- 172.21.15.7:0/2545001031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22103 (secure 0 0 0) 0x7fb8240bd0d0 con 0x7fb834074960 2024-04-20T21:31:02.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:02.972 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:30:53.904734+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:02.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.973+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 msgr2=0x7fb8180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.973+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 0x7fb8180a6180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb82002c210 tx=0x7fb820034b70 comp rx=0 tx=0).stop 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.973+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 msgr2=0x7fb83419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.973+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb82400e9f0 tx=0x7fb82400edf0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 shutdown_connections 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb8180a3cc0 0x7fb8180a6180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb834109e60 0x7fb8341a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb834075f40 0x7fb83419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 --2- 172.21.15.7:0/2545001031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb834074960 0x7fb83419a490 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 >> 172.21.15.7:0/2545001031 conn(0x7fb8340fd540 msgr2=0x7fb834105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 shutdown_connections 2024-04-20T21:31:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:02.974+0000 7fb83b400640 1 -- 172.21.15.7:0/2545001031 wait complete. 2024-04-20T21:31:02.992 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:31:02.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:03.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.199+0000 7f76e2000640 1 -- 172.21.15.7:0/1746538216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc077b60 msgr2=0x7f76dc077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.199+0000 7f76e2000640 1 --2- 172.21.15.7:0/1746538216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc077b60 0x7f76dc077fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009900 tx=0x7f76c402bc30 comp rx=0 tx=0).stop 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 -- 172.21.15.7:0/1746538216 shutdown_connections 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 --2- 172.21.15.7:0/1746538216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc077b60 0x7f76dc077fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 --2- 172.21.15.7:0/1746538216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc078500 0x7f76dc078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 --2- 172.21.15.7:0/1746538216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 0x7f76dc076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 -- 172.21.15.7:0/1746538216 >> 172.21.15.7:0/1746538216 conn(0x7f76dc0ff7b0 msgr2=0x7f76dc101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 -- 172.21.15.7:0/1746538216 shutdown_connections 2024-04-20T21:31:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.200+0000 7f76e2000640 1 -- 172.21.15.7:0/1746538216 wait complete. 2024-04-20T21:31:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.201+0000 7f76e2000640 1 Processor -- start 2024-04-20T21:31:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.201+0000 7f76e2000640 1 -- start start 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 0x7f76dc19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc078500 0x7f76dc1a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc1142a0 con 0x7f76dc077b60 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc114120 con 0x7f76dc078500 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76e2000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc114420 con 0x7f76dc076920 2024-04-20T21:31:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33574/0 (socket says 172.21.15.7:33574) 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 -- 172.21.15.7:0/4017008428 learned_addr learned my addr 172.21.15.7:0/4017008428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76dbe00640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc078500 0x7f76dc1a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76db400640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 0x7f76dc19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 -- 172.21.15.7:0/4017008428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 msgr2=0x7f76dc19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 0x7f76dc19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 -- 172.21.15.7:0/4017008428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc078500 msgr2=0x7f76dc1a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc078500 0x7f76dc1a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.202+0000 7f76d3e00640 1 -- 172.21.15.7:0/4017008428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc1a64e0 con 0x7f76dc077b60 2024-04-20T21:31:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76d3e00640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76c4000c00 tx=0x7f76c402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4043020 con 0x7f76dc077b60 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc1a6770 con 0x7f76dc077b60 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76c40054e0 con 0x7f76dc077b60 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4003bd0 con 0x7f76dc077b60 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.203+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc1a6cc0 con 0x7f76dc077b60 2024-04-20T21:31:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.204+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc110d40 con 0x7f76dc077b60 2024-04-20T21:31:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.206+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f76c4046020 con 0x7f76dc077b60 2024-04-20T21:31:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.208+0000 7f76d8c00640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 0x7f76c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.208+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f76c40f2460 con 0x7f76dc077b60 2024-04-20T21:31:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.208+0000 7f76db400640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 0x7f76c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.209+0000 7f76db400640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 0x7f76c00a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009bd0 tx=0x7f76cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.212+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f76c404d020 con 0x7f76dc077b60 2024-04-20T21:31:03.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:03 smithi007 ceph-mon[16867]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:03 smithi007 ceph-mon[16867]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:03 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/825265955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:03 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2545001031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:03 smithi186 ceph-mon[22157]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:03 smithi186 ceph-mon[22157]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:03 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/825265955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.366 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:03 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2545001031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.419+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76dc0623a0 con 0x7f76c00a3c40 2024-04-20T21:31:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.435+0000 7f76d8c00640 1 -- 172.21.15.7:0/4017008428 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7f76dc0623a0 con 0x7f76c00a3c40 2024-04-20T21:31:03.440 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.441+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 msgr2=0x7f76c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.441+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 0x7f76c00a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009bd0 tx=0x7f76cc005f90 comp rx=0 tx=0).stop 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.441+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 msgr2=0x7f76dc19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.441+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76c4000c00 tx=0x7f76c402c650 comp rx=0 tx=0).stop 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 shutdown_connections 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f76c00a3c40 0x7f76c00a6100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc078500 0x7f76dc1a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc077b60 0x7f76dc19ee00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 --2- 172.21.15.7:0/4017008428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f76dc076920 0x7f76dc19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 >> 172.21.15.7:0/4017008428 conn(0x7f76dc0ff7b0 msgr2=0x7f76dc100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 shutdown_connections 2024-04-20T21:31:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.442+0000 7f76e2000640 1 -- 172.21.15.7:0/4017008428 wait complete. 2024-04-20T21:31:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:03 smithi016 ceph-mon[21409]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:03 smithi016 ceph-mon[21409]: from='client.24689 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:31:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:03 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/825265955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:03 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2545001031' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:03.463 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-20T21:31:02.356129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595500,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":300,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351972,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589852,"statfs":{"total":1151940427776,"available":1151580008448,"internally_reserved":0,"allocated":11403264,"data_stored":7082840,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5132,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":119,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.450244"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551142+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:30:54.551142+0000","last_peered":"2024-04-20T21:30:54.551142+0000","last_clean":"2024-04-20T21:30:54.551142+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:30:54.551142+0000","last_undegraded":"2024-04-20T21:30:54.551142+0000","last_fullsized":"2024-04-20T21:30:54.551142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920453+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:30:53.920453+0000","last_peered":"2024-04-20T21:30:53.920453+0000","last_clean":"2024-04-20T21:30:53.920453+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:30:53.920453+0000","last_undegraded":"2024-04-20T21:30:53.920453+0000","last_fullsized":"2024-04-20T21:30:53.920453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920357+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:30:53.920357+0000","last_peered":"2024-04-20T21:30:53.920357+0000","last_clean":"2024-04-20T21:30:53.920357+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:30:53.920357+0000","last_undegraded":"2024-04-20T21:30:53.920357+0000","last_fullsized":"2024-04-20T21:30:53.920357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910057+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:30:53.910057+0000","last_peered":"2024-04-20T21:30:53.910057+0000","last_clean":"2024-04-20T21:30:53.910057+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:30:53.910057+0000","last_undegraded":"2024-04-20T21:30:53.910057+0000","last_fullsized":"2024-04-20T21:30:53.910057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988970+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:30:53.988970+0000","last_peered":"2024-04-20T21:30:53.988970+0000","last_clean":"2024-04-20T21:30:53.988970+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:30:53.988970+0000","last_undegraded":"2024-04-20T21:30:53.988970+0000","last_fullsized":"2024-04-20T21:30:53.988970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989980+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:30:53.989980+0000","last_peered":"2024-04-20T21:30:53.989980+0000","last_clean":"2024-04-20T21:30:53.989980+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:30:53.989980+0000","last_undegraded":"2024-04-20T21:30:53.989980+0000","last_fullsized":"2024-04-20T21:30:53.989980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989910+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:30:53.989910+0000","last_peered":"2024-04-20T21:30:53.989910+0000","last_clean":"2024-04-20T21:30:53.989910+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:30:53.989910+0000","last_undegraded":"2024-04-20T21:30:53.989910+0000","last_fullsized":"2024-04-20T21:30:53.989910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909555+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:30:53.909555+0000","last_peered":"2024-04-20T21:30:53.909555+0000","last_clean":"2024-04-20T21:30:53.909555+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:30:53.909555+0000","last_undegraded":"2024-04-20T21:30:53.909555+0000","last_fullsized":"2024-04-20T21:30:53.909555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990625+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:30:53.990625+0000","last_peered":"2024-04-20T21:30:53.990625+0000","last_clean":"2024-04-20T21:30:53.990625+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:30:53.990625+0000","last_undegraded":"2024-04-20T21:30:53.990625+0000","last_fullsized":"2024-04-20T21:30:53.990625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990715+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:30:53.990715+0000","last_peered":"2024-04-20T21:30:53.990715+0000","last_clean":"2024-04-20T21:30:53.990715+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:30:53.990715+0000","last_undegraded":"2024-04-20T21:30:53.990715+0000","last_fullsized":"2024-04-20T21:30:53.990715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990583+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:30:53.990583+0000","last_peered":"2024-04-20T21:30:53.990583+0000","last_clean":"2024-04-20T21:30:53.990583+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:30:53.990583+0000","last_undegraded":"2024-04-20T21:30:53.990583+0000","last_fullsized":"2024-04-20T21:30:53.990583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910078+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:30:53.910078+0000","last_peered":"2024-04-20T21:30:53.910078+0000","last_clean":"2024-04-20T21:30:53.910078+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:30:53.910078+0000","last_undegraded":"2024-04-20T21:30:53.910078+0000","last_fullsized":"2024-04-20T21:30:53.910078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990236+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:30:53.990236+0000","last_peered":"2024-04-20T21:30:53.990236+0000","last_clean":"2024-04-20T21:30:53.990236+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:30:53.990236+0000","last_undegraded":"2024-04-20T21:30:53.990236+0000","last_fullsized":"2024-04-20T21:30:53.990236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920444+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:30:53.920444+0000","last_peered":"2024-04-20T21:30:53.920444+0000","last_clean":"2024-04-20T21:30:53.920444+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:30:53.920444+0000","last_undegraded":"2024-04-20T21:30:53.920444+0000","last_fullsized":"2024-04-20T21:30:53.920444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920146+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:30:53.920146+0000","last_peered":"2024-04-20T21:30:53.920146+0000","last_clean":"2024-04-20T21:30:53.920146+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:30:53.920146+0000","last_undegraded":"2024-04-20T21:30:53.920146+0000","last_fullsized":"2024-04-20T21:30:53.920146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989326+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:30:53.989326+0000","last_peered":"2024-04-20T21:30:53.989326+0000","last_clean":"2024-04-20T21:30:53.989326+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:30:53.989326+0000","last_undegraded":"2024-04-20T21:30:53.989326+0000","last_fullsized":"2024-04-20T21:30:53.989326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551261+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:30:54.551261+0000","last_peered":"2024-04-20T21:30:54.551261+0000","last_clean":"2024-04-20T21:30:54.551261+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:30:54.551261+0000","last_undegraded":"2024-04-20T21:30:54.551261+0000","last_fullsized":"2024-04-20T21:30:54.551261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.957345+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:30:53.957345+0000","last_peered":"2024-04-20T21:30:53.957345+0000","last_clean":"2024-04-20T21:30:53.957345+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:30:53.957345+0000","last_undegraded":"2024-04-20T21:30:53.957345+0000","last_fullsized":"2024-04-20T21:30:53.957345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989833+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:30:53.989833+0000","last_peered":"2024-04-20T21:30:53.989833+0000","last_clean":"2024-04-20T21:30:53.989833+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:30:53.989833+0000","last_undegraded":"2024-04-20T21:30:53.989833+0000","last_fullsized":"2024-04-20T21:30:53.989833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.964706+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:30:57.964706+0000","last_peered":"2024-04-20T21:30:57.964706+0000","last_clean":"2024-04-20T21:30:57.964706+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:30:57.964706+0000","last_undegraded":"2024-04-20T21:30:57.964706+0000","last_fullsized":"2024-04-20T21:30:57.964706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910181+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:30:53.910181+0000","last_peered":"2024-04-20T21:30:53.910181+0000","last_clean":"2024-04-20T21:30:53.910181+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:30:53.910181+0000","last_undegraded":"2024-04-20T21:30:53.910181+0000","last_fullsized":"2024-04-20T21:30:53.910181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:55.489105+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:30:55.489105+0000","last_peered":"2024-04-20T21:30:55.489105+0000","last_clean":"2024-04-20T21:30:55.489105+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:30:55.489105+0000","last_undegraded":"2024-04-20T21:30:55.489105+0000","last_fullsized":"2024-04-20T21:30:55.489105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920736+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:30:53.920736+0000","last_peered":"2024-04-20T21:30:53.920736+0000","last_clean":"2024-04-20T21:30:53.920736+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:30:53.920736+0000","last_undegraded":"2024-04-20T21:30:53.920736+0000","last_fullsized":"2024-04-20T21:30:53.920736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957307+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:30:57.957307+0000","last_peered":"2024-04-20T21:30:57.957307+0000","last_clean":"2024-04-20T21:30:57.957307+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:30:57.957307+0000","last_undegraded":"2024-04-20T21:30:57.957307+0000","last_fullsized":"2024-04-20T21:30:57.957307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989945+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:30:53.989945+0000","last_peered":"2024-04-20T21:30:53.989945+0000","last_clean":"2024-04-20T21:30:53.989945+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:30:53.989945+0000","last_undegraded":"2024-04-20T21:30:53.989945+0000","last_fullsized":"2024-04-20T21:30:53.989945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989239+0000","last_change":"2024-04-20T21:30:52.380190+0000","last_active":"2024-04-20T21:30:53.989239+0000","last_peered":"2024-04-20T21:30:53.989239+0000","last_clean":"2024-04-20T21:30:53.989239+0000","last_became_active":"2024-04-20T21:30:52.379884+0000","last_became_peered":"2024-04-20T21:30:52.379884+0000","last_unstale":"2024-04-20T21:30:53.989239+0000","last_undegraded":"2024-04-20T21:30:53.989239+0000","last_fullsized":"2024-04-20T21:30:53.989239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:02.585806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910393+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:30:53.910393+0000","last_peered":"2024-04-20T21:30:53.910393+0000","last_clean":"2024-04-20T21:30:53.910393+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:30:53.910393+0000","last_undegraded":"2024-04-20T21:30:53.910393+0000","last_fullsized":"2024-04-20T21:30:53.910393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938942+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:30:56.938942+0000","last_peered":"2024-04-20T21:30:56.938942+0000","last_clean":"2024-04-20T21:30:56.938942+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:30:56.938942+0000","last_undegraded":"2024-04-20T21:30:56.938942+0000","last_fullsized":"2024-04-20T21:30:56.938942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988906+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:30:53.988906+0000","last_peered":"2024-04-20T21:30:53.988906+0000","last_clean":"2024-04-20T21:30:53.988906+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:30:53.988906+0000","last_undegraded":"2024-04-20T21:30:53.988906+0000","last_fullsized":"2024-04-20T21:30:53.988906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.963412+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:30:57.963412+0000","last_peered":"2024-04-20T21:30:57.963412+0000","last_clean":"2024-04-20T21:30:57.963412+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:30:57.963412+0000","last_undegraded":"2024-04-20T21:30:57.963412+0000","last_fullsized":"2024-04-20T21:30:57.963412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910182+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:30:53.910182+0000","last_peered":"2024-04-20T21:30:53.910182+0000","last_clean":"2024-04-20T21:30:53.910182+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:30:53.910182+0000","last_undegraded":"2024-04-20T21:30:53.910182+0000","last_fullsized":"2024-04-20T21:30:53.910182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989853+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:30:53.989853+0000","last_peered":"2024-04-20T21:30:53.989853+0000","last_clean":"2024-04-20T21:30:53.989853+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:30:53.989853+0000","last_undegraded":"2024-04-20T21:30:53.989853+0000","last_fullsized":"2024-04-20T21:30:53.989853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909952+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:30:53.909952+0000","last_peered":"2024-04-20T21:30:53.909952+0000","last_clean":"2024-04-20T21:30:53.909952+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:30:53.909952+0000","last_undegraded":"2024-04-20T21:30:53.909952+0000","last_fullsized":"2024-04-20T21:30:53.909952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909937+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:30:53.909937+0000","last_peered":"2024-04-20T21:30:53.909937+0000","last_clean":"2024-04-20T21:30:53.909937+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:30:53.909937+0000","last_undegraded":"2024-04-20T21:30:53.909937+0000","last_fullsized":"2024-04-20T21:30:53.909937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.956985+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:30:53.956985+0000","last_peered":"2024-04-20T21:30:53.956985+0000","last_clean":"2024-04-20T21:30:53.956985+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:30:53.956985+0000","last_undegraded":"2024-04-20T21:30:53.956985+0000","last_fullsized":"2024-04-20T21:30:53.956985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.940346+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:30:56.940346+0000","last_peered":"2024-04-20T21:30:56.940346+0000","last_clean":"2024-04-20T21:30:56.940346+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:30:56.940346+0000","last_undegraded":"2024-04-20T21:30:56.940346+0000","last_fullsized":"2024-04-20T21:30:56.940346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989819+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:30:53.989819+0000","last_peered":"2024-04-20T21:30:53.989819+0000","last_clean":"2024-04-20T21:30:53.989819+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:30:53.989819+0000","last_undegraded":"2024-04-20T21:30:53.989819+0000","last_fullsized":"2024-04-20T21:30:53.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938588+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:30:56.938588+0000","last_peered":"2024-04-20T21:30:56.938588+0000","last_clean":"2024-04-20T21:30:56.938588+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:30:56.938588+0000","last_undegraded":"2024-04-20T21:30:56.938588+0000","last_fullsized":"2024-04-20T21:30:56.938588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920656+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:30:53.920656+0000","last_peered":"2024-04-20T21:30:53.920656+0000","last_clean":"2024-04-20T21:30:53.920656+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:30:53.920656+0000","last_undegraded":"2024-04-20T21:30:53.920656+0000","last_fullsized":"2024-04-20T21:30:53.920656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910462+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:30:53.910462+0000","last_peered":"2024-04-20T21:30:53.910462+0000","last_clean":"2024-04-20T21:30:53.910462+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:30:53.910462+0000","last_undegraded":"2024-04-20T21:30:53.910462+0000","last_fullsized":"2024-04-20T21:30:53.910462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990288+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:30:53.990288+0000","last_peered":"2024-04-20T21:30:53.990288+0000","last_clean":"2024-04-20T21:30:53.990288+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:30:53.990288+0000","last_undegraded":"2024-04-20T21:30:53.990288+0000","last_fullsized":"2024-04-20T21:30:53.990288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:58.409625+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:30:58.409625+0000","last_peered":"2024-04-20T21:30:58.409625+0000","last_clean":"2024-04-20T21:30:58.409625+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:30:58.409625+0000","last_undegraded":"2024-04-20T21:30:58.409625+0000","last_fullsized":"2024-04-20T21:30:58.409625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989527+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:30:53.989527+0000","last_peered":"2024-04-20T21:30:53.989527+0000","last_clean":"2024-04-20T21:30:53.989527+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:30:53.989527+0000","last_undegraded":"2024-04-20T21:30:53.989527+0000","last_fullsized":"2024-04-20T21:30:53.989527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.962025+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:30:57.962025+0000","last_peered":"2024-04-20T21:30:57.962025+0000","last_clean":"2024-04-20T21:30:57.962025+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:30:57.962025+0000","last_undegraded":"2024-04-20T21:30:57.962025+0000","last_fullsized":"2024-04-20T21:30:57.962025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989695+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:30:53.989695+0000","last_peered":"2024-04-20T21:30:53.989695+0000","last_clean":"2024-04-20T21:30:53.989695+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:30:53.989695+0000","last_undegraded":"2024-04-20T21:30:53.989695+0000","last_fullsized":"2024-04-20T21:30:53.989695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.941217+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:30:56.941217+0000","last_peered":"2024-04-20T21:30:56.941217+0000","last_clean":"2024-04-20T21:30:56.941217+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:30:56.941217+0000","last_undegraded":"2024-04-20T21:30:56.941217+0000","last_fullsized":"2024-04-20T21:30:56.941217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991058+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:30:53.991058+0000","last_peered":"2024-04-20T21:30:53.991058+0000","last_clean":"2024-04-20T21:30:53.991058+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:30:53.991058+0000","last_undegraded":"2024-04-20T21:30:53.991058+0000","last_fullsized":"2024-04-20T21:30:53.991058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957871+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:30:57.957871+0000","last_peered":"2024-04-20T21:30:57.957871+0000","last_clean":"2024-04-20T21:30:57.957871+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:30:57.957871+0000","last_undegraded":"2024-04-20T21:30:57.957871+0000","last_fullsized":"2024-04-20T21:30:57.957871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989495+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:30:53.989495+0000","last_peered":"2024-04-20T21:30:53.989495+0000","last_clean":"2024-04-20T21:30:53.989495+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:30:53.989495+0000","last_undegraded":"2024-04-20T21:30:53.989495+0000","last_fullsized":"2024-04-20T21:30:53.989495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.967536+0000","last_change":"2024-04-20T21:30:52.379933+0000","last_active":"2024-04-20T21:30:57.967536+0000","last_peered":"2024-04-20T21:30:57.967536+0000","last_clean":"2024-04-20T21:30:57.967536+0000","last_became_active":"2024-04-20T21:30:52.379585+0000","last_became_peered":"2024-04-20T21:30:52.379585+0000","last_unstale":"2024-04-20T21:30:57.967536+0000","last_undegraded":"2024-04-20T21:30:57.967536+0000","last_fullsized":"2024-04-20T21:30:57.967536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:52:47.643441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909967+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:30:53.909967+0000","last_peered":"2024-04-20T21:30:53.909967+0000","last_clean":"2024-04-20T21:30:53.909967+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:30:53.909967+0000","last_undegraded":"2024-04-20T21:30:53.909967+0000","last_fullsized":"2024-04-20T21:30:53.909967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:58.407420+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:30:58.407420+0000","last_peered":"2024-04-20T21:30:58.407420+0000","last_clean":"2024-04-20T21:30:58.407420+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:30:58.407420+0000","last_undegraded":"2024-04-20T21:30:58.407420+0000","last_fullsized":"2024-04-20T21:30:58.407420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551213+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:30:54.551213+0000","last_peered":"2024-04-20T21:30:54.551213+0000","last_clean":"2024-04-20T21:30:54.551213+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:30:54.551213+0000","last_undegraded":"2024-04-20T21:30:54.551213+0000","last_fullsized":"2024-04-20T21:30:54.551213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.957003+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:30:53.957003+0000","last_peered":"2024-04-20T21:30:53.957003+0000","last_clean":"2024-04-20T21:30:53.957003+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:30:53.957003+0000","last_undegraded":"2024-04-20T21:30:53.957003+0000","last_fullsized":"2024-04-20T21:30:53.957003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990595+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:30:53.990595+0000","last_peered":"2024-04-20T21:30:53.990595+0000","last_clean":"2024-04-20T21:30:53.990595+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:30:53.990595+0000","last_undegraded":"2024-04-20T21:30:53.990595+0000","last_fullsized":"2024-04-20T21:30:53.990595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.684005+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:30:54.684005+0000","last_peered":"2024-04-20T21:30:54.684005+0000","last_clean":"2024-04-20T21:30:54.684005+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:30:54.684005+0000","last_undegraded":"2024-04-20T21:30:54.684005+0000","last_fullsized":"2024-04-20T21:30:54.684005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990040+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:30:53.990040+0000","last_peered":"2024-04-20T21:30:53.990040+0000","last_clean":"2024-04-20T21:30:53.990040+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:30:53.990040+0000","last_undegraded":"2024-04-20T21:30:53.990040+0000","last_fullsized":"2024-04-20T21:30:53.990040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910177+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:30:53.910177+0000","last_peered":"2024-04-20T21:30:53.910177+0000","last_clean":"2024-04-20T21:30:53.910177+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:30:53.910177+0000","last_undegraded":"2024-04-20T21:30:53.910177+0000","last_fullsized":"2024-04-20T21:30:53.910177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990300+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:30:53.990300+0000","last_peered":"2024-04-20T21:30:53.990300+0000","last_clean":"2024-04-20T21:30:53.990300+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:30:53.990300+0000","last_undegraded":"2024-04-20T21:30:53.990300+0000","last_fullsized":"2024-04-20T21:30:53.990300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909743+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:30:53.909743+0000","last_peered":"2024-04-20T21:30:53.909743+0000","last_clean":"2024-04-20T21:30:53.909743+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:30:53.909743+0000","last_undegraded":"2024-04-20T21:30:53.909743+0000","last_fullsized":"2024-04-20T21:30:53.909743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.939917+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:30:56.939917+0000","last_peered":"2024-04-20T21:30:56.939917+0000","last_clean":"2024-04-20T21:30:56.939917+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:30:56.939917+0000","last_undegraded":"2024-04-20T21:30:56.939917+0000","last_fullsized":"2024-04-20T21:30:56.939917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990480+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:30:53.990480+0000","last_peered":"2024-04-20T21:30:53.990480+0000","last_clean":"2024-04-20T21:30:53.990480+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:30:53.990480+0000","last_undegraded":"2024-04-20T21:30:53.990480+0000","last_fullsized":"2024-04-20T21:30:53.990480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989571+0000","last_change":"2024-04-20T21:30:52.934254+0000","last_active":"2024-04-20T21:30:53.989571+0000","last_peered":"2024-04-20T21:30:53.989571+0000","last_clean":"2024-04-20T21:30:53.989571+0000","last_became_active":"2024-04-20T21:30:52.933905+0000","last_became_peered":"2024-04-20T21:30:52.933905+0000","last_unstale":"2024-04-20T21:30:53.989571+0000","last_undegraded":"2024-04-20T21:30:53.989571+0000","last_fullsized":"2024-04-20T21:30:53.989571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:01:02.750118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957229+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:30:57.957229+0000","last_peered":"2024-04-20T21:30:57.957229+0000","last_clean":"2024-04-20T21:30:57.957229+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:30:57.957229+0000","last_undegraded":"2024-04-20T21:30:57.957229+0000","last_fullsized":"2024-04-20T21:30:57.957229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938928+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:30:56.938928+0000","last_peered":"2024-04-20T21:30:56.938928+0000","last_clean":"2024-04-20T21:30:56.938928+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:30:56.938928+0000","last_undegraded":"2024-04-20T21:30:56.938928+0000","last_fullsized":"2024-04-20T21:30:56.938928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989820+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:30:53.989820+0000","last_peered":"2024-04-20T21:30:53.989820+0000","last_clean":"2024-04-20T21:30:53.989820+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:30:53.989820+0000","last_undegraded":"2024-04-20T21:30:53.989820+0000","last_fullsized":"2024-04-20T21:30:53.989820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.966215+0000","last_change":"2024-04-20T21:30:52.397308+0000","last_active":"2024-04-20T21:30:57.966215+0000","last_peered":"2024-04-20T21:30:57.966215+0000","last_clean":"2024-04-20T21:30:57.966215+0000","last_became_active":"2024-04-20T21:30:52.396933+0000","last_became_peered":"2024-04-20T21:30:52.396933+0000","last_unstale":"2024-04-20T21:30:57.966215+0000","last_undegraded":"2024-04-20T21:30:57.966215+0000","last_fullsized":"2024-04-20T21:30:57.966215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:05.786009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910480+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:30:53.910480+0000","last_peered":"2024-04-20T21:30:53.910480+0000","last_clean":"2024-04-20T21:30:53.910480+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:30:53.910480+0000","last_undegraded":"2024-04-20T21:30:53.910480+0000","last_fullsized":"2024-04-20T21:30:53.910480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.936834+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:30:56.936834+0000","last_peered":"2024-04-20T21:30:56.936834+0000","last_clean":"2024-04-20T21:30:56.936834+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:30:56.936834+0000","last_undegraded":"2024-04-20T21:30:56.936834+0000","last_fullsized":"2024-04-20T21:30:56.936834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920685+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:30:53.920685+0000","last_peered":"2024-04-20T21:30:53.920685+0000","last_clean":"2024-04-20T21:30:53.920685+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:30:53.920685+0000","last_undegraded":"2024-04-20T21:30:53.920685+0000","last_fullsized":"2024-04-20T21:30:53.920685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.939989+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:30:56.939989+0000","last_peered":"2024-04-20T21:30:56.939989+0000","last_clean":"2024-04-20T21:30:56.939989+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:30:56.939989+0000","last_undegraded":"2024-04-20T21:30:56.939989+0000","last_fullsized":"2024-04-20T21:30:56.939989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988926+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:30:53.988926+0000","last_peered":"2024-04-20T21:30:53.988926+0000","last_clean":"2024-04-20T21:30:53.988926+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:30:53.988926+0000","last_undegraded":"2024-04-20T21:30:53.988926+0000","last_fullsized":"2024-04-20T21:30:53.988926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.941557+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:30:56.941557+0000","last_peered":"2024-04-20T21:30:56.941557+0000","last_clean":"2024-04-20T21:30:56.941557+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:30:56.941557+0000","last_undegraded":"2024-04-20T21:30:56.941557+0000","last_fullsized":"2024-04-20T21:30:56.941557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.550880+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:30:54.550880+0000","last_peered":"2024-04-20T21:30:54.550880+0000","last_clean":"2024-04-20T21:30:54.550880+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:30:54.550880+0000","last_undegraded":"2024-04-20T21:30:54.550880+0000","last_fullsized":"2024-04-20T21:30:54.550880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988943+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:30:53.988943+0000","last_peered":"2024-04-20T21:30:53.988943+0000","last_clean":"2024-04-20T21:30:53.988943+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:30:53.988943+0000","last_undegraded":"2024-04-20T21:30:53.988943+0000","last_fullsized":"2024-04-20T21:30:53.988943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909985+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:30:53.909985+0000","last_peered":"2024-04-20T21:30:53.909985+0000","last_clean":"2024-04-20T21:30:53.909985+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:30:53.909985+0000","last_undegraded":"2024-04-20T21:30:53.909985+0000","last_fullsized":"2024-04-20T21:30:53.909985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.937230+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:30:56.937230+0000","last_peered":"2024-04-20T21:30:56.937230+0000","last_clean":"2024-04-20T21:30:56.937230+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:30:56.937230+0000","last_undegraded":"2024-04-20T21:30:56.937230+0000","last_fullsized":"2024-04-20T21:30:56.937230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991009+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:30:53.991009+0000","last_peered":"2024-04-20T21:30:53.991009+0000","last_clean":"2024-04-20T21:30:53.991009+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:30:53.991009+0000","last_undegraded":"2024-04-20T21:30:53.991009+0000","last_fullsized":"2024-04-20T21:30:53.991009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.967424+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:30:57.967424+0000","last_peered":"2024-04-20T21:30:57.967424+0000","last_clean":"2024-04-20T21:30:57.967424+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:30:57.967424+0000","last_undegraded":"2024-04-20T21:30:57.967424+0000","last_fullsized":"2024-04-20T21:30:57.967424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990785+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:30:53.990785+0000","last_peered":"2024-04-20T21:30:53.990785+0000","last_clean":"2024-04-20T21:30:53.990785+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:30:53.990785+0000","last_undegraded":"2024-04-20T21:30:53.990785+0000","last_fullsized":"2024-04-20T21:30:53.990785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910090+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:30:53.910090+0000","last_peered":"2024-04-20T21:30:53.910090+0000","last_clean":"2024-04-20T21:30:53.910090+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:30:53.910090+0000","last_undegraded":"2024-04-20T21:30:53.910090+0000","last_fullsized":"2024-04-20T21:30:53.910090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990167+0000","last_change":"2024-04-20T21:30:52.942340+0000","last_active":"2024-04-20T21:30:53.990167+0000","last_peered":"2024-04-20T21:30:53.990167+0000","last_clean":"2024-04-20T21:30:53.990167+0000","last_became_active":"2024-04-20T21:30:52.941782+0000","last_became_peered":"2024-04-20T21:30:52.941782+0000","last_unstale":"2024-04-20T21:30:53.990167+0000","last_undegraded":"2024-04-20T21:30:53.990167+0000","last_fullsized":"2024-04-20T21:30:53.990167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:31:47.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989973+0000","last_change":"2024-04-20T21:30:52.399733+0000","last_active":"2024-04-20T21:30:53.989973+0000","last_peered":"2024-04-20T21:30:53.989973+0000","last_clean":"2024-04-20T21:30:53.989973+0000","last_became_active":"2024-04-20T21:30:52.399384+0000","last_became_peered":"2024-04-20T21:30:52.399384+0000","last_unstale":"2024-04-20T21:30:53.989973+0000","last_undegraded":"2024-04-20T21:30:53.989973+0000","last_fullsized":"2024-04-20T21:30:53.989973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:42.097722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991124+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:30:53.991124+0000","last_peered":"2024-04-20T21:30:53.991124+0000","last_clean":"2024-04-20T21:30:53.991124+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:30:53.991124+0000","last_undegraded":"2024-04-20T21:30:53.991124+0000","last_fullsized":"2024-04-20T21:30:53.991124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551001+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:30:54.551001+0000","last_peered":"2024-04-20T21:30:54.551001+0000","last_clean":"2024-04-20T21:30:54.551001+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:30:54.551001+0000","last_undegraded":"2024-04-20T21:30:54.551001+0000","last_fullsized":"2024-04-20T21:30:54.551001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989606+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:30:53.989606+0000","last_peered":"2024-04-20T21:30:53.989606+0000","last_clean":"2024-04-20T21:30:53.989606+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:30:53.989606+0000","last_undegraded":"2024-04-20T21:30:53.989606+0000","last_fullsized":"2024-04-20T21:30:53.989606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989531+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:30:53.989531+0000","last_peered":"2024-04-20T21:30:53.989531+0000","last_clean":"2024-04-20T21:30:53.989531+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:30:53.989531+0000","last_undegraded":"2024-04-20T21:30:53.989531+0000","last_fullsized":"2024-04-20T21:30:53.989531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920397+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:30:53.920397+0000","last_peered":"2024-04-20T21:30:53.920397+0000","last_clean":"2024-04-20T21:30:53.920397+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:30:53.920397+0000","last_undegraded":"2024-04-20T21:30:53.920397+0000","last_fullsized":"2024-04-20T21:30:53.920397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989819+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:30:53.989819+0000","last_peered":"2024-04-20T21:30:53.989819+0000","last_clean":"2024-04-20T21:30:53.989819+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:30:53.989819+0000","last_undegraded":"2024-04-20T21:30:53.989819+0000","last_fullsized":"2024-04-20T21:30:53.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551163+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:30:54.551163+0000","last_peered":"2024-04-20T21:30:54.551163+0000","last_clean":"2024-04-20T21:30:54.551163+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:30:54.551163+0000","last_undegraded":"2024-04-20T21:30:54.551163+0000","last_fullsized":"2024-04-20T21:30:54.551163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989838+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:30:53.989838+0000","last_peered":"2024-04-20T21:30:53.989838+0000","last_clean":"2024-04-20T21:30:53.989838+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:30:53.989838+0000","last_undegraded":"2024-04-20T21:30:53.989838+0000","last_fullsized":"2024-04-20T21:30:53.989838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910234+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:30:53.910234+0000","last_peered":"2024-04-20T21:30:53.910234+0000","last_clean":"2024-04-20T21:30:53.910234+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:30:53.910234+0000","last_undegraded":"2024-04-20T21:30:53.910234+0000","last_fullsized":"2024-04-20T21:30:53.910234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910141+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:30:53.910141+0000","last_peered":"2024-04-20T21:30:53.910141+0000","last_clean":"2024-04-20T21:30:53.910141+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:30:53.910141+0000","last_undegraded":"2024-04-20T21:30:53.910141+0000","last_fullsized":"2024-04-20T21:30:53.910141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909716+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:30:53.909716+0000","last_peered":"2024-04-20T21:30:53.909716+0000","last_clean":"2024-04-20T21:30:53.909716+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:30:53.909716+0000","last_undegraded":"2024-04-20T21:30:53.909716+0000","last_fullsized":"2024-04-20T21:30:53.909716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990191+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:30:53.990191+0000","last_peered":"2024-04-20T21:30:53.990191+0000","last_clean":"2024-04-20T21:30:53.990191+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:30:53.990191+0000","last_undegraded":"2024-04-20T21:30:53.990191+0000","last_fullsized":"2024-04-20T21:30:53.990191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990128+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:30:53.990128+0000","last_peered":"2024-04-20T21:30:53.990128+0000","last_clean":"2024-04-20T21:30:53.990128+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:30:53.990128+0000","last_undegraded":"2024-04-20T21:30:53.990128+0000","last_fullsized":"2024-04-20T21:30:53.990128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551208+0000","last_change":"2024-04-20T21:30:52.933302+0000","last_active":"2024-04-20T21:30:54.551208+0000","last_peered":"2024-04-20T21:30:54.551208+0000","last_clean":"2024-04-20T21:30:54.551208+0000","last_became_active":"2024-04-20T21:30:52.932856+0000","last_became_peered":"2024-04-20T21:30:52.932856+0000","last_unstale":"2024-04-20T21:30:54.551208+0000","last_undegraded":"2024-04-20T21:30:54.551208+0000","last_fullsized":"2024-04-20T21:30:54.551208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:54:10.226142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281618,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5132,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":119,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":14792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1033732,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:03.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 -- 172.21.15.7:0/1223170491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10102fe0 msgr2=0x7f7b10103460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 --2- 172.21.15.7:0/1223170491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10102fe0 0x7f7b10103460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009900 tx=0x7f7af802bc40 comp rx=0 tx=0).stop 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 -- 172.21.15.7:0/1223170491 shutdown_connections 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 --2- 172.21.15.7:0/1223170491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b101039a0 0x7f7b1010a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 --2- 172.21.15.7:0/1223170491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10102fe0 0x7f7b10103460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 --2- 172.21.15.7:0/1223170491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b10101e90 0x7f7b101022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 -- 172.21.15.7:0/1223170491 >> 172.21.15.7:0/1223170491 conn(0x7f7b100fd420 msgr2=0x7f7b100ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 -- 172.21.15.7:0/1223170491 shutdown_connections 2024-04-20T21:31:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.724+0000 7f7b15200640 1 -- 172.21.15.7:0/1223170491 wait complete. 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 Processor -- start 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 -- start start 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 0x7f7b1019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b101039a0 0x7f7b101a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010db20 con 0x7f7b10101e90 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010d9a0 con 0x7f7b10102fe0 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.725+0000 7f7b15200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010dca0 con 0x7f7b101039a0 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33596/0 (socket says 172.21.15.7:33596) 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 -- 172.21.15.7:0/3044874239 learned_addr learned my addr 172.21.15.7:0/3044874239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0fe00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b101039a0 0x7f7b101a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b07e00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 0x7f7b1019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 -- 172.21.15.7:0/3044874239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b101039a0 msgr2=0x7f7b101a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b101039a0 0x7f7b101a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 -- 172.21.15.7:0/3044874239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 msgr2=0x7f7b1019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 0x7f7b1019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 -- 172.21.15.7:0/3044874239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101a1ff0 con 0x7f7b10101e90 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b07e00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 0x7f7b1019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.726+0000 7f7b0f400640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000e9f0 tx=0x7f7b0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.727+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000cd80 con 0x7f7b10101e90 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.727+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b00010910 con 0x7f7b10101e90 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.727+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00018990 con 0x7f7b10101e90 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.727+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a22e0 con 0x7f7b10101e90 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.727+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a27b0 con 0x7f7b10101e90 2024-04-20T21:31:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.728+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b101a2a10 con 0x7f7b10101e90 2024-04-20T21:31:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.730+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f7b00010ac0 con 0x7f7b10101e90 2024-04-20T21:31:03.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.733+0000 7f7b0cc00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 0x7f7af40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.733+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7b000d08b0 con 0x7f7b10101e90 2024-04-20T21:31:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.733+0000 7f7b07e00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 0x7f7af40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.734+0000 7f7b07e00640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 0x7f7af40a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7af802c7c0 tx=0x7f7af8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.737+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7b0009d1e0 con 0x7f7b10101e90 2024-04-20T21:31:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.944+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b100758b0 con 0x7f7af40a3c30 2024-04-20T21:31:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.957+0000 7f7b0cc00640 1 -- 172.21.15.7:0/3044874239 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258892 (secure 0 0 0) 0x7f7b100758b0 con 0x7f7af40a3c30 2024-04-20T21:31:03.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:31:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 msgr2=0x7f7af40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 0x7f7af40a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7af802c7c0 tx=0x7f7af8005fd0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 msgr2=0x7f7b1019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000e9f0 tx=0x7f7b0000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 shutdown_connections 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7af40a3c30 0x7f7af40a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b10102fe0 0x7f7b1019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b101039a0 0x7f7b101a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 --2- 172.21.15.7:0/3044874239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b10101e90 0x7f7b1019a3d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 >> 172.21.15.7:0/3044874239 conn(0x7f7b100fd420 msgr2=0x7f7b10074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 shutdown_connections 2024-04-20T21:31:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:03.963+0000 7f7b15200640 1 -- 172.21.15.7:0/3044874239 wait complete. 2024-04-20T21:31:03.985 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-20T21:31:02.356129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595500,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":147,"num_read_kb":129,"num_write":300,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":351972,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589852,"statfs":{"total":1151940427776,"available":1151580008448,"internally_reserved":0,"allocated":11403264,"data_stored":7082840,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5132,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":119,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.450244"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551142+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:30:54.551142+0000","last_peered":"2024-04-20T21:30:54.551142+0000","last_clean":"2024-04-20T21:30:54.551142+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:30:54.551142+0000","last_undegraded":"2024-04-20T21:30:54.551142+0000","last_fullsized":"2024-04-20T21:30:54.551142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920453+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:30:53.920453+0000","last_peered":"2024-04-20T21:30:53.920453+0000","last_clean":"2024-04-20T21:30:53.920453+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:30:53.920453+0000","last_undegraded":"2024-04-20T21:30:53.920453+0000","last_fullsized":"2024-04-20T21:30:53.920453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920357+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:30:53.920357+0000","last_peered":"2024-04-20T21:30:53.920357+0000","last_clean":"2024-04-20T21:30:53.920357+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:30:53.920357+0000","last_undegraded":"2024-04-20T21:30:53.920357+0000","last_fullsized":"2024-04-20T21:30:53.920357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910057+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:30:53.910057+0000","last_peered":"2024-04-20T21:30:53.910057+0000","last_clean":"2024-04-20T21:30:53.910057+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:30:53.910057+0000","last_undegraded":"2024-04-20T21:30:53.910057+0000","last_fullsized":"2024-04-20T21:30:53.910057+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988970+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:30:53.988970+0000","last_peered":"2024-04-20T21:30:53.988970+0000","last_clean":"2024-04-20T21:30:53.988970+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:30:53.988970+0000","last_undegraded":"2024-04-20T21:30:53.988970+0000","last_fullsized":"2024-04-20T21:30:53.988970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989980+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:30:53.989980+0000","last_peered":"2024-04-20T21:30:53.989980+0000","last_clean":"2024-04-20T21:30:53.989980+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:30:53.989980+0000","last_undegraded":"2024-04-20T21:30:53.989980+0000","last_fullsized":"2024-04-20T21:30:53.989980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989910+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:30:53.989910+0000","last_peered":"2024-04-20T21:30:53.989910+0000","last_clean":"2024-04-20T21:30:53.989910+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:30:53.989910+0000","last_undegraded":"2024-04-20T21:30:53.989910+0000","last_fullsized":"2024-04-20T21:30:53.989910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909555+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:30:53.909555+0000","last_peered":"2024-04-20T21:30:53.909555+0000","last_clean":"2024-04-20T21:30:53.909555+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:30:53.909555+0000","last_undegraded":"2024-04-20T21:30:53.909555+0000","last_fullsized":"2024-04-20T21:30:53.909555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990625+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:30:53.990625+0000","last_peered":"2024-04-20T21:30:53.990625+0000","last_clean":"2024-04-20T21:30:53.990625+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:30:53.990625+0000","last_undegraded":"2024-04-20T21:30:53.990625+0000","last_fullsized":"2024-04-20T21:30:53.990625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990715+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:30:53.990715+0000","last_peered":"2024-04-20T21:30:53.990715+0000","last_clean":"2024-04-20T21:30:53.990715+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:30:53.990715+0000","last_undegraded":"2024-04-20T21:30:53.990715+0000","last_fullsized":"2024-04-20T21:30:53.990715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990583+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:30:53.990583+0000","last_peered":"2024-04-20T21:30:53.990583+0000","last_clean":"2024-04-20T21:30:53.990583+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:30:53.990583+0000","last_undegraded":"2024-04-20T21:30:53.990583+0000","last_fullsized":"2024-04-20T21:30:53.990583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910078+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:30:53.910078+0000","last_peered":"2024-04-20T21:30:53.910078+0000","last_clean":"2024-04-20T21:30:53.910078+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:30:53.910078+0000","last_undegraded":"2024-04-20T21:30:53.910078+0000","last_fullsized":"2024-04-20T21:30:53.910078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990236+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:30:53.990236+0000","last_peered":"2024-04-20T21:30:53.990236+0000","last_clean":"2024-04-20T21:30:53.990236+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:30:53.990236+0000","last_undegraded":"2024-04-20T21:30:53.990236+0000","last_fullsized":"2024-04-20T21:30:53.990236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920444+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:30:53.920444+0000","last_peered":"2024-04-20T21:30:53.920444+0000","last_clean":"2024-04-20T21:30:53.920444+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:30:53.920444+0000","last_undegraded":"2024-04-20T21:30:53.920444+0000","last_fullsized":"2024-04-20T21:30:53.920444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920146+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:30:53.920146+0000","last_peered":"2024-04-20T21:30:53.920146+0000","last_clean":"2024-04-20T21:30:53.920146+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:30:53.920146+0000","last_undegraded":"2024-04-20T21:30:53.920146+0000","last_fullsized":"2024-04-20T21:30:53.920146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989326+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:30:53.989326+0000","last_peered":"2024-04-20T21:30:53.989326+0000","last_clean":"2024-04-20T21:30:53.989326+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:30:53.989326+0000","last_undegraded":"2024-04-20T21:30:53.989326+0000","last_fullsized":"2024-04-20T21:30:53.989326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551261+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:30:54.551261+0000","last_peered":"2024-04-20T21:30:54.551261+0000","last_clean":"2024-04-20T21:30:54.551261+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:30:54.551261+0000","last_undegraded":"2024-04-20T21:30:54.551261+0000","last_fullsized":"2024-04-20T21:30:54.551261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.957345+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:30:53.957345+0000","last_peered":"2024-04-20T21:30:53.957345+0000","last_clean":"2024-04-20T21:30:53.957345+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:30:53.957345+0000","last_undegraded":"2024-04-20T21:30:53.957345+0000","last_fullsized":"2024-04-20T21:30:53.957345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989833+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:30:53.989833+0000","last_peered":"2024-04-20T21:30:53.989833+0000","last_clean":"2024-04-20T21:30:53.989833+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:30:53.989833+0000","last_undegraded":"2024-04-20T21:30:53.989833+0000","last_fullsized":"2024-04-20T21:30:53.989833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.964706+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:30:57.964706+0000","last_peered":"2024-04-20T21:30:57.964706+0000","last_clean":"2024-04-20T21:30:57.964706+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:30:57.964706+0000","last_undegraded":"2024-04-20T21:30:57.964706+0000","last_fullsized":"2024-04-20T21:30:57.964706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910181+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:30:53.910181+0000","last_peered":"2024-04-20T21:30:53.910181+0000","last_clean":"2024-04-20T21:30:53.910181+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:30:53.910181+0000","last_undegraded":"2024-04-20T21:30:53.910181+0000","last_fullsized":"2024-04-20T21:30:53.910181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:55.489105+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:30:55.489105+0000","last_peered":"2024-04-20T21:30:55.489105+0000","last_clean":"2024-04-20T21:30:55.489105+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:30:55.489105+0000","last_undegraded":"2024-04-20T21:30:55.489105+0000","last_fullsized":"2024-04-20T21:30:55.489105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920736+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:30:53.920736+0000","last_peered":"2024-04-20T21:30:53.920736+0000","last_clean":"2024-04-20T21:30:53.920736+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:30:53.920736+0000","last_undegraded":"2024-04-20T21:30:53.920736+0000","last_fullsized":"2024-04-20T21:30:53.920736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957307+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:30:57.957307+0000","last_peered":"2024-04-20T21:30:57.957307+0000","last_clean":"2024-04-20T21:30:57.957307+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:30:57.957307+0000","last_undegraded":"2024-04-20T21:30:57.957307+0000","last_fullsized":"2024-04-20T21:30:57.957307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989945+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:30:53.989945+0000","last_peered":"2024-04-20T21:30:53.989945+0000","last_clean":"2024-04-20T21:30:53.989945+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:30:53.989945+0000","last_undegraded":"2024-04-20T21:30:53.989945+0000","last_fullsized":"2024-04-20T21:30:53.989945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989239+0000","last_change":"2024-04-20T21:30:52.380190+0000","last_active":"2024-04-20T21:30:53.989239+0000","last_peered":"2024-04-20T21:30:53.989239+0000","last_clean":"2024-04-20T21:30:53.989239+0000","last_became_active":"2024-04-20T21:30:52.379884+0000","last_became_peered":"2024-04-20T21:30:52.379884+0000","last_unstale":"2024-04-20T21:30:53.989239+0000","last_undegraded":"2024-04-20T21:30:53.989239+0000","last_fullsized":"2024-04-20T21:30:53.989239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:02.585806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910393+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:30:53.910393+0000","last_peered":"2024-04-20T21:30:53.910393+0000","last_clean":"2024-04-20T21:30:53.910393+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:30:53.910393+0000","last_undegraded":"2024-04-20T21:30:53.910393+0000","last_fullsized":"2024-04-20T21:30:53.910393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938942+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:30:56.938942+0000","last_peered":"2024-04-20T21:30:56.938942+0000","last_clean":"2024-04-20T21:30:56.938942+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:30:56.938942+0000","last_undegraded":"2024-04-20T21:30:56.938942+0000","last_fullsized":"2024-04-20T21:30:56.938942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988906+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:30:53.988906+0000","last_peered":"2024-04-20T21:30:53.988906+0000","last_clean":"2024-04-20T21:30:53.988906+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:30:53.988906+0000","last_undegraded":"2024-04-20T21:30:53.988906+0000","last_fullsized":"2024-04-20T21:30:53.988906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.963412+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:30:57.963412+0000","last_peered":"2024-04-20T21:30:57.963412+0000","last_clean":"2024-04-20T21:30:57.963412+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:30:57.963412+0000","last_undegraded":"2024-04-20T21:30:57.963412+0000","last_fullsized":"2024-04-20T21:30:57.963412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910182+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:30:53.910182+0000","last_peered":"2024-04-20T21:30:53.910182+0000","last_clean":"2024-04-20T21:30:53.910182+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:30:53.910182+0000","last_undegraded":"2024-04-20T21:30:53.910182+0000","last_fullsized":"2024-04-20T21:30:53.910182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989853+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:30:53.989853+0000","last_peered":"2024-04-20T21:30:53.989853+0000","last_clean":"2024-04-20T21:30:53.989853+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:30:53.989853+0000","last_undegraded":"2024-04-20T21:30:53.989853+0000","last_fullsized":"2024-04-20T21:30:53.989853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909952+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:30:53.909952+0000","last_peered":"2024-04-20T21:30:53.909952+0000","last_clean":"2024-04-20T21:30:53.909952+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:30:53.909952+0000","last_undegraded":"2024-04-20T21:30:53.909952+0000","last_fullsized":"2024-04-20T21:30:53.909952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909937+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:30:53.909937+0000","last_peered":"2024-04-20T21:30:53.909937+0000","last_clean":"2024-04-20T21:30:53.909937+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:30:53.909937+0000","last_undegraded":"2024-04-20T21:30:53.909937+0000","last_fullsized":"2024-04-20T21:30:53.909937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.956985+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:30:53.956985+0000","last_peered":"2024-04-20T21:30:53.956985+0000","last_clean":"2024-04-20T21:30:53.956985+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:30:53.956985+0000","last_undegraded":"2024-04-20T21:30:53.956985+0000","last_fullsized":"2024-04-20T21:30:53.956985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.940346+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:30:56.940346+0000","last_peered":"2024-04-20T21:30:56.940346+0000","last_clean":"2024-04-20T21:30:56.940346+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:30:56.940346+0000","last_undegraded":"2024-04-20T21:30:56.940346+0000","last_fullsized":"2024-04-20T21:30:56.940346+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989819+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:30:53.989819+0000","last_peered":"2024-04-20T21:30:53.989819+0000","last_clean":"2024-04-20T21:30:53.989819+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:30:53.989819+0000","last_undegraded":"2024-04-20T21:30:53.989819+0000","last_fullsized":"2024-04-20T21:30:53.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938588+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:30:56.938588+0000","last_peered":"2024-04-20T21:30:56.938588+0000","last_clean":"2024-04-20T21:30:56.938588+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:30:56.938588+0000","last_undegraded":"2024-04-20T21:30:56.938588+0000","last_fullsized":"2024-04-20T21:30:56.938588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920656+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:30:53.920656+0000","last_peered":"2024-04-20T21:30:53.920656+0000","last_clean":"2024-04-20T21:30:53.920656+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:30:53.920656+0000","last_undegraded":"2024-04-20T21:30:53.920656+0000","last_fullsized":"2024-04-20T21:30:53.920656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910462+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:30:53.910462+0000","last_peered":"2024-04-20T21:30:53.910462+0000","last_clean":"2024-04-20T21:30:53.910462+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:30:53.910462+0000","last_undegraded":"2024-04-20T21:30:53.910462+0000","last_fullsized":"2024-04-20T21:30:53.910462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990288+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:30:53.990288+0000","last_peered":"2024-04-20T21:30:53.990288+0000","last_clean":"2024-04-20T21:30:53.990288+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:30:53.990288+0000","last_undegraded":"2024-04-20T21:30:53.990288+0000","last_fullsized":"2024-04-20T21:30:53.990288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:58.409625+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:30:58.409625+0000","last_peered":"2024-04-20T21:30:58.409625+0000","last_clean":"2024-04-20T21:30:58.409625+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:30:58.409625+0000","last_undegraded":"2024-04-20T21:30:58.409625+0000","last_fullsized":"2024-04-20T21:30:58.409625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989527+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:30:53.989527+0000","last_peered":"2024-04-20T21:30:53.989527+0000","last_clean":"2024-04-20T21:30:53.989527+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:30:53.989527+0000","last_undegraded":"2024-04-20T21:30:53.989527+0000","last_fullsized":"2024-04-20T21:30:53.989527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.962025+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:30:57.962025+0000","last_peered":"2024-04-20T21:30:57.962025+0000","last_clean":"2024-04-20T21:30:57.962025+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:30:57.962025+0000","last_undegraded":"2024-04-20T21:30:57.962025+0000","last_fullsized":"2024-04-20T21:30:57.962025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989695+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:30:53.989695+0000","last_peered":"2024-04-20T21:30:53.989695+0000","last_clean":"2024-04-20T21:30:53.989695+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:30:53.989695+0000","last_undegraded":"2024-04-20T21:30:53.989695+0000","last_fullsized":"2024-04-20T21:30:53.989695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.941217+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:30:56.941217+0000","last_peered":"2024-04-20T21:30:56.941217+0000","last_clean":"2024-04-20T21:30:56.941217+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:30:56.941217+0000","last_undegraded":"2024-04-20T21:30:56.941217+0000","last_fullsized":"2024-04-20T21:30:56.941217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991058+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:30:53.991058+0000","last_peered":"2024-04-20T21:30:53.991058+0000","last_clean":"2024-04-20T21:30:53.991058+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:30:53.991058+0000","last_undegraded":"2024-04-20T21:30:53.991058+0000","last_fullsized":"2024-04-20T21:30:53.991058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957871+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:30:57.957871+0000","last_peered":"2024-04-20T21:30:57.957871+0000","last_clean":"2024-04-20T21:30:57.957871+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:30:57.957871+0000","last_undegraded":"2024-04-20T21:30:57.957871+0000","last_fullsized":"2024-04-20T21:30:57.957871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989495+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:30:53.989495+0000","last_peered":"2024-04-20T21:30:53.989495+0000","last_clean":"2024-04-20T21:30:53.989495+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:30:53.989495+0000","last_undegraded":"2024-04-20T21:30:53.989495+0000","last_fullsized":"2024-04-20T21:30:53.989495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.967536+0000","last_change":"2024-04-20T21:30:52.379933+0000","last_active":"2024-04-20T21:30:57.967536+0000","last_peered":"2024-04-20T21:30:57.967536+0000","last_clean":"2024-04-20T21:30:57.967536+0000","last_became_active":"2024-04-20T21:30:52.379585+0000","last_became_peered":"2024-04-20T21:30:52.379585+0000","last_unstale":"2024-04-20T21:30:57.967536+0000","last_undegraded":"2024-04-20T21:30:57.967536+0000","last_fullsized":"2024-04-20T21:30:57.967536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:52:47.643441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909967+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:30:53.909967+0000","last_peered":"2024-04-20T21:30:53.909967+0000","last_clean":"2024-04-20T21:30:53.909967+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:30:53.909967+0000","last_undegraded":"2024-04-20T21:30:53.909967+0000","last_fullsized":"2024-04-20T21:30:53.909967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:58.407420+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:30:58.407420+0000","last_peered":"2024-04-20T21:30:58.407420+0000","last_clean":"2024-04-20T21:30:58.407420+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:30:58.407420+0000","last_undegraded":"2024-04-20T21:30:58.407420+0000","last_fullsized":"2024-04-20T21:30:58.407420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551213+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:30:54.551213+0000","last_peered":"2024-04-20T21:30:54.551213+0000","last_clean":"2024-04-20T21:30:54.551213+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:30:54.551213+0000","last_undegraded":"2024-04-20T21:30:54.551213+0000","last_fullsized":"2024-04-20T21:30:54.551213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.957003+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:30:53.957003+0000","last_peered":"2024-04-20T21:30:53.957003+0000","last_clean":"2024-04-20T21:30:53.957003+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:30:53.957003+0000","last_undegraded":"2024-04-20T21:30:53.957003+0000","last_fullsized":"2024-04-20T21:30:53.957003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990595+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:30:53.990595+0000","last_peered":"2024-04-20T21:30:53.990595+0000","last_clean":"2024-04-20T21:30:53.990595+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:30:53.990595+0000","last_undegraded":"2024-04-20T21:30:53.990595+0000","last_fullsized":"2024-04-20T21:30:53.990595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.684005+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:30:54.684005+0000","last_peered":"2024-04-20T21:30:54.684005+0000","last_clean":"2024-04-20T21:30:54.684005+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:30:54.684005+0000","last_undegraded":"2024-04-20T21:30:54.684005+0000","last_fullsized":"2024-04-20T21:30:54.684005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990040+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:30:53.990040+0000","last_peered":"2024-04-20T21:30:53.990040+0000","last_clean":"2024-04-20T21:30:53.990040+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:30:53.990040+0000","last_undegraded":"2024-04-20T21:30:53.990040+0000","last_fullsized":"2024-04-20T21:30:53.990040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910177+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:30:53.910177+0000","last_peered":"2024-04-20T21:30:53.910177+0000","last_clean":"2024-04-20T21:30:53.910177+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:30:53.910177+0000","last_undegraded":"2024-04-20T21:30:53.910177+0000","last_fullsized":"2024-04-20T21:30:53.910177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990300+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:30:53.990300+0000","last_peered":"2024-04-20T21:30:53.990300+0000","last_clean":"2024-04-20T21:30:53.990300+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:30:53.990300+0000","last_undegraded":"2024-04-20T21:30:53.990300+0000","last_fullsized":"2024-04-20T21:30:53.990300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909743+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:30:53.909743+0000","last_peered":"2024-04-20T21:30:53.909743+0000","last_clean":"2024-04-20T21:30:53.909743+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:30:53.909743+0000","last_undegraded":"2024-04-20T21:30:53.909743+0000","last_fullsized":"2024-04-20T21:30:53.909743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.939917+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:30:56.939917+0000","last_peered":"2024-04-20T21:30:56.939917+0000","last_clean":"2024-04-20T21:30:56.939917+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:30:56.939917+0000","last_undegraded":"2024-04-20T21:30:56.939917+0000","last_fullsized":"2024-04-20T21:30:56.939917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990480+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:30:53.990480+0000","last_peered":"2024-04-20T21:30:53.990480+0000","last_clean":"2024-04-20T21:30:53.990480+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:30:53.990480+0000","last_undegraded":"2024-04-20T21:30:53.990480+0000","last_fullsized":"2024-04-20T21:30:53.990480+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989571+0000","last_change":"2024-04-20T21:30:52.934254+0000","last_active":"2024-04-20T21:30:53.989571+0000","last_peered":"2024-04-20T21:30:53.989571+0000","last_clean":"2024-04-20T21:30:53.989571+0000","last_became_active":"2024-04-20T21:30:52.933905+0000","last_became_peered":"2024-04-20T21:30:52.933905+0000","last_unstale":"2024-04-20T21:30:53.989571+0000","last_undegraded":"2024-04-20T21:30:53.989571+0000","last_fullsized":"2024-04-20T21:30:53.989571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:01:02.750118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.957229+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:30:57.957229+0000","last_peered":"2024-04-20T21:30:57.957229+0000","last_clean":"2024-04-20T21:30:57.957229+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:30:57.957229+0000","last_undegraded":"2024-04-20T21:30:57.957229+0000","last_fullsized":"2024-04-20T21:30:57.957229+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.938928+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:30:56.938928+0000","last_peered":"2024-04-20T21:30:56.938928+0000","last_clean":"2024-04-20T21:30:56.938928+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:30:56.938928+0000","last_undegraded":"2024-04-20T21:30:56.938928+0000","last_fullsized":"2024-04-20T21:30:56.938928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989820+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:30:53.989820+0000","last_peered":"2024-04-20T21:30:53.989820+0000","last_clean":"2024-04-20T21:30:53.989820+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:30:53.989820+0000","last_undegraded":"2024-04-20T21:30:53.989820+0000","last_fullsized":"2024-04-20T21:30:53.989820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.966215+0000","last_change":"2024-04-20T21:30:52.397308+0000","last_active":"2024-04-20T21:30:57.966215+0000","last_peered":"2024-04-20T21:30:57.966215+0000","last_clean":"2024-04-20T21:30:57.966215+0000","last_became_active":"2024-04-20T21:30:52.396933+0000","last_became_peered":"2024-04-20T21:30:52.396933+0000","last_unstale":"2024-04-20T21:30:57.966215+0000","last_undegraded":"2024-04-20T21:30:57.966215+0000","last_fullsized":"2024-04-20T21:30:57.966215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:05.786009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910480+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:30:53.910480+0000","last_peered":"2024-04-20T21:30:53.910480+0000","last_clean":"2024-04-20T21:30:53.910480+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:30:53.910480+0000","last_undegraded":"2024-04-20T21:30:53.910480+0000","last_fullsized":"2024-04-20T21:30:53.910480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.936834+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:30:56.936834+0000","last_peered":"2024-04-20T21:30:56.936834+0000","last_clean":"2024-04-20T21:30:56.936834+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:30:56.936834+0000","last_undegraded":"2024-04-20T21:30:56.936834+0000","last_fullsized":"2024-04-20T21:30:56.936834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920685+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:30:53.920685+0000","last_peered":"2024-04-20T21:30:53.920685+0000","last_clean":"2024-04-20T21:30:53.920685+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:30:53.920685+0000","last_undegraded":"2024-04-20T21:30:53.920685+0000","last_fullsized":"2024-04-20T21:30:53.920685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.939989+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:30:56.939989+0000","last_peered":"2024-04-20T21:30:56.939989+0000","last_clean":"2024-04-20T21:30:56.939989+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:30:56.939989+0000","last_undegraded":"2024-04-20T21:30:56.939989+0000","last_fullsized":"2024-04-20T21:30:56.939989+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988926+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:30:53.988926+0000","last_peered":"2024-04-20T21:30:53.988926+0000","last_clean":"2024-04-20T21:30:53.988926+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:30:53.988926+0000","last_undegraded":"2024-04-20T21:30:53.988926+0000","last_fullsized":"2024-04-20T21:30:53.988926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.941557+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:30:56.941557+0000","last_peered":"2024-04-20T21:30:56.941557+0000","last_clean":"2024-04-20T21:30:56.941557+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:30:56.941557+0000","last_undegraded":"2024-04-20T21:30:56.941557+0000","last_fullsized":"2024-04-20T21:30:56.941557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.550880+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:30:54.550880+0000","last_peered":"2024-04-20T21:30:54.550880+0000","last_clean":"2024-04-20T21:30:54.550880+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:30:54.550880+0000","last_undegraded":"2024-04-20T21:30:54.550880+0000","last_fullsized":"2024-04-20T21:30:54.550880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.988943+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:30:53.988943+0000","last_peered":"2024-04-20T21:30:53.988943+0000","last_clean":"2024-04-20T21:30:53.988943+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:30:53.988943+0000","last_undegraded":"2024-04-20T21:30:53.988943+0000","last_fullsized":"2024-04-20T21:30:53.988943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909985+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:30:53.909985+0000","last_peered":"2024-04-20T21:30:53.909985+0000","last_clean":"2024-04-20T21:30:53.909985+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:30:53.909985+0000","last_undegraded":"2024-04-20T21:30:53.909985+0000","last_fullsized":"2024-04-20T21:30:53.909985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:56.937230+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:30:56.937230+0000","last_peered":"2024-04-20T21:30:56.937230+0000","last_clean":"2024-04-20T21:30:56.937230+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:30:56.937230+0000","last_undegraded":"2024-04-20T21:30:56.937230+0000","last_fullsized":"2024-04-20T21:30:56.937230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991009+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:30:53.991009+0000","last_peered":"2024-04-20T21:30:53.991009+0000","last_clean":"2024-04-20T21:30:53.991009+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:30:53.991009+0000","last_undegraded":"2024-04-20T21:30:53.991009+0000","last_fullsized":"2024-04-20T21:30:53.991009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:57.967424+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:30:57.967424+0000","last_peered":"2024-04-20T21:30:57.967424+0000","last_clean":"2024-04-20T21:30:57.967424+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:30:57.967424+0000","last_undegraded":"2024-04-20T21:30:57.967424+0000","last_fullsized":"2024-04-20T21:30:57.967424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990785+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:30:53.990785+0000","last_peered":"2024-04-20T21:30:53.990785+0000","last_clean":"2024-04-20T21:30:53.990785+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:30:53.990785+0000","last_undegraded":"2024-04-20T21:30:53.990785+0000","last_fullsized":"2024-04-20T21:30:53.990785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910090+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:30:53.910090+0000","last_peered":"2024-04-20T21:30:53.910090+0000","last_clean":"2024-04-20T21:30:53.910090+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:30:53.910090+0000","last_undegraded":"2024-04-20T21:30:53.910090+0000","last_fullsized":"2024-04-20T21:30:53.910090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990167+0000","last_change":"2024-04-20T21:30:52.942340+0000","last_active":"2024-04-20T21:30:53.990167+0000","last_peered":"2024-04-20T21:30:53.990167+0000","last_clean":"2024-04-20T21:30:53.990167+0000","last_became_active":"2024-04-20T21:30:52.941782+0000","last_became_peered":"2024-04-20T21:30:52.941782+0000","last_unstale":"2024-04-20T21:30:53.990167+0000","last_undegraded":"2024-04-20T21:30:53.990167+0000","last_fullsized":"2024-04-20T21:30:53.990167+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:31:47.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989973+0000","last_change":"2024-04-20T21:30:52.399733+0000","last_active":"2024-04-20T21:30:53.989973+0000","last_peered":"2024-04-20T21:30:53.989973+0000","last_clean":"2024-04-20T21:30:53.989973+0000","last_became_active":"2024-04-20T21:30:52.399384+0000","last_became_peered":"2024-04-20T21:30:52.399384+0000","last_unstale":"2024-04-20T21:30:53.989973+0000","last_undegraded":"2024-04-20T21:30:53.989973+0000","last_fullsized":"2024-04-20T21:30:53.989973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:42.097722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.991124+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:30:53.991124+0000","last_peered":"2024-04-20T21:30:53.991124+0000","last_clean":"2024-04-20T21:30:53.991124+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:30:53.991124+0000","last_undegraded":"2024-04-20T21:30:53.991124+0000","last_fullsized":"2024-04-20T21:30:53.991124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551001+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:30:54.551001+0000","last_peered":"2024-04-20T21:30:54.551001+0000","last_clean":"2024-04-20T21:30:54.551001+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:30:54.551001+0000","last_undegraded":"2024-04-20T21:30:54.551001+0000","last_fullsized":"2024-04-20T21:30:54.551001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989606+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:30:53.989606+0000","last_peered":"2024-04-20T21:30:53.989606+0000","last_clean":"2024-04-20T21:30:53.989606+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:30:53.989606+0000","last_undegraded":"2024-04-20T21:30:53.989606+0000","last_fullsized":"2024-04-20T21:30:53.989606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989531+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:30:53.989531+0000","last_peered":"2024-04-20T21:30:53.989531+0000","last_clean":"2024-04-20T21:30:53.989531+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:30:53.989531+0000","last_undegraded":"2024-04-20T21:30:53.989531+0000","last_fullsized":"2024-04-20T21:30:53.989531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.920397+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:30:53.920397+0000","last_peered":"2024-04-20T21:30:53.920397+0000","last_clean":"2024-04-20T21:30:53.920397+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:30:53.920397+0000","last_undegraded":"2024-04-20T21:30:53.920397+0000","last_fullsized":"2024-04-20T21:30:53.920397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989819+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:30:53.989819+0000","last_peered":"2024-04-20T21:30:53.989819+0000","last_clean":"2024-04-20T21:30:53.989819+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:30:53.989819+0000","last_undegraded":"2024-04-20T21:30:53.989819+0000","last_fullsized":"2024-04-20T21:30:53.989819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551163+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:30:54.551163+0000","last_peered":"2024-04-20T21:30:54.551163+0000","last_clean":"2024-04-20T21:30:54.551163+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:30:54.551163+0000","last_undegraded":"2024-04-20T21:30:54.551163+0000","last_fullsized":"2024-04-20T21:30:54.551163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.989838+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:30:53.989838+0000","last_peered":"2024-04-20T21:30:53.989838+0000","last_clean":"2024-04-20T21:30:53.989838+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:30:53.989838+0000","last_undegraded":"2024-04-20T21:30:53.989838+0000","last_fullsized":"2024-04-20T21:30:53.989838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910234+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:30:53.910234+0000","last_peered":"2024-04-20T21:30:53.910234+0000","last_clean":"2024-04-20T21:30:53.910234+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:30:53.910234+0000","last_undegraded":"2024-04-20T21:30:53.910234+0000","last_fullsized":"2024-04-20T21:30:53.910234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.910141+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:30:53.910141+0000","last_peered":"2024-04-20T21:30:53.910141+0000","last_clean":"2024-04-20T21:30:53.910141+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:30:53.910141+0000","last_undegraded":"2024-04-20T21:30:53.910141+0000","last_fullsized":"2024-04-20T21:30:53.910141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.909716+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:30:53.909716+0000","last_peered":"2024-04-20T21:30:53.909716+0000","last_clean":"2024-04-20T21:30:53.909716+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:30:53.909716+0000","last_undegraded":"2024-04-20T21:30:53.909716+0000","last_fullsized":"2024-04-20T21:30:53.909716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990191+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:30:53.990191+0000","last_peered":"2024-04-20T21:30:53.990191+0000","last_clean":"2024-04-20T21:30:53.990191+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:30:53.990191+0000","last_undegraded":"2024-04-20T21:30:53.990191+0000","last_fullsized":"2024-04-20T21:30:53.990191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:53.990128+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:30:53.990128+0000","last_peered":"2024-04-20T21:30:53.990128+0000","last_clean":"2024-04-20T21:30:53.990128+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:30:53.990128+0000","last_undegraded":"2024-04-20T21:30:53.990128+0000","last_fullsized":"2024-04-20T21:30:53.990128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-20T21:30:54.551208+0000","last_change":"2024-04-20T21:30:52.933302+0000","last_active":"2024-04-20T21:30:54.551208+0000","last_peered":"2024-04-20T21:30:54.551208+0000","last_clean":"2024-04-20T21:30:54.551208+0000","last_became_active":"2024-04-20T21:30:52.932856+0000","last_became_peered":"2024-04-20T21:30:52.932856+0000","last_unstale":"2024-04-20T21:30:54.551208+0000","last_undegraded":"2024-04-20T21:30:54.551208+0000","last_fullsized":"2024-04-20T21:30:54.551208+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:54:10.226142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281618,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5132,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":119,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":14792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035130,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":442310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":1033732,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":441856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":1033416,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:03.989 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:31:03.990 INFO:tasks.cephadm:Setting up client nodes... 2024-04-20T21:31:03.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-20T21:31:04.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:04 smithi007 ceph-mon[16867]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-20T21:31:04.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:04 smithi016 ceph-mon[21409]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-20T21:31:04.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:04 smithi186 ceph-mon[22157]: pgmap v323: 97 pgs: 97 active+clean; 582 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-20T21:31:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:05.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:05 smithi007 ceph-mon[16867]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:05.252 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:05 smithi007 ceph-mon[16867]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:05.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:05 smithi016 ceph-mon[21409]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:05.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:05 smithi016 ceph-mon[21409]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:05.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:05 smithi186 ceph-mon[22157]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:05.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:05 smithi186 ceph-mon[22157]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:31:06.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:06 smithi007 ceph-mon[16867]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-20T21:31:06.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:06 smithi016 ceph-mon[21409]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-20T21:31:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:06 smithi186 ceph-mon[22157]: pgmap v324: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-04-20T21:31:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 -- 172.21.15.7:0/2649307354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394102ad0 msgr2=0x7f9394102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 --2- 172.21.15.7:0/2649307354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394102ad0 0x7f9394102ef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f93880099c0 tx=0x7f9388009cd0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 -- 172.21.15.7:0/2649307354 shutdown_connections 2024-04-20T21:31:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 --2- 172.21.15.7:0/2649307354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394104670 0x7f939410af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 --2- 172.21.15.7:0/2649307354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394102ad0 0x7f9394102ef0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 --2- 172.21.15.7:0/2649307354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394103cd0 0x7f9394104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 -- 172.21.15.7:0/2649307354 >> 172.21.15.7:0/2649307354 conn(0x7f93940fe070 msgr2=0x7f93941004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 -- 172.21.15.7:0/2649307354 shutdown_connections 2024-04-20T21:31:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.000+0000 7f939b400700 1 -- 172.21.15.7:0/2649307354 wait complete. 2024-04-20T21:31:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.002+0000 7f939b400700 1 Processor -- start 2024-04-20T21:31:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.002+0000 7f939b400700 1 -- start start 2024-04-20T21:31:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 0x7f9394074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394104670 0x7f9394075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e690 con 0x7f9394104670 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e510 con 0x7f9394103cd0 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.003+0000 7f939b400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f939410e810 con 0x7f9394102ad0 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45222/0 (socket says 172.21.15.7:45222) 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 -- 172.21.15.7:0/1824760719 learned_addr learned my addr 172.21.15.7:0/1824760719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9393e00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 0x7f9394074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9399400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394104670 0x7f9394075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 -- 172.21.15.7:0/1824760719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 msgr2=0x7f9394074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 0x7f9394074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 -- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394104670 msgr2=0x7f9394075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394104670 0x7f9394075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9398a00700 1 -- 172.21.15.7:0/1824760719 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9394075c30 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.004+0000 7f9393e00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 0x7f9394074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f9398a00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9388003790 tx=0x7f9388009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388009e60 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a2d10 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388003e40 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388005690 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.005+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a31c0 con 0x7f9394102ad0 2024-04-20T21:31:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.006+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394050f90 con 0x7f9394102ad0 2024-04-20T21:31:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.008+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f938800fca0 con 0x7f9394102ad0 2024-04-20T21:31:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.010+0000 7f9391600700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 0x7f937c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.010+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93880d13b0 con 0x7f9394102ad0 2024-04-20T21:31:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.010+0000 7f9393e00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 0x7f937c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.011+0000 7f9393e00700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 0x7f937c0a6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9380011490 tx=0x7f9380009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.015+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9388013070 con 0x7f9394102ad0 2024-04-20T21:31:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.333+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9394063660 con 0x7f9394102ad0 2024-04-20T21:31:07.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.341+0000 7f9391600700 1 -- 172.21.15.7:0/1824760719 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f938809dd70 con 0x7f9394102ad0 2024-04-20T21:31:07.341 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-20T21:31:07.341 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAbNCRmFJUXFBAAvo3w9AQtnc/cXuCRzoxQ2Q== 2024-04-20T21:31:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 msgr2=0x7f937c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 0x7f937c0a6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9380011490 tx=0x7f9380009260 comp rx=0 tx=0).stop 2024-04-20T21:31:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 msgr2=0x7f93940767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9388003790 tx=0x7f9388009580 comp rx=0 tx=0).stop 2024-04-20T21:31:07.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 shutdown_connections 2024-04-20T21:31:07.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f937c0a3d80 0x7f937c0a6240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9394102ad0 0x7f93940767f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9394104670 0x7f9394075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 --2- 172.21.15.7:0/1824760719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9394103cd0 0x7f9394074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 >> 172.21.15.7:0/1824760719 conn(0x7f93940fe070 msgr2=0x7f9394106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 shutdown_connections 2024-04-20T21:31:07.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:07.344+0000 7f939b400700 1 -- 172.21.15.7:0/1824760719 wait complete. 2024-04-20T21:31:08.122 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:31:08.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-20T21:31:08.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T21:31:08.268 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T21:31:08.268 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-20T21:31:08.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-20T21:31:08.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:08 smithi007 ceph-mon[16867]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-20T21:31:08.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1824760719' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:08 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:08 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:08.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:08 smithi016 ceph-mon[21409]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-20T21:31:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1824760719' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:08 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:08 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:08.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:08 smithi186 ceph-mon[22157]: pgmap v325: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2024-04-20T21:31:08.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1824760719' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:08 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-20T21:31:08.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:08 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-20T21:31:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:10.196 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:10 smithi007 ceph-mon[16867]: pgmap v326: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-20T21:31:10.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:10 smithi016 ceph-mon[21409]: pgmap v326: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-20T21:31:10.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:10 smithi186 ceph-mon[22157]: pgmap v326: 97 pgs: 97 active+clean; 620 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.7 KiB/s wr, 15 op/s 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 -- 172.21.15.7:0/3179809220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2334108f90 msgr2=0x7f2334109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 --2- 172.21.15.7:0/3179809220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2334108f90 0x7f2334109410 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2320009990 tx=0x7f2320009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 -- 172.21.15.7:0/3179809220 shutdown_connections 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 --2- 172.21.15.7:0/3179809220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 0x7f2334115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 --2- 172.21.15.7:0/3179809220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233410f040 0x7f233410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 --2- 172.21.15.7:0/3179809220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2334108f90 0x7f2334109410 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 -- 172.21.15.7:0/3179809220 >> 172.21.15.7:0/3179809220 conn(0x7f23340787e0 msgr2=0x7f2334107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 -- 172.21.15.7:0/3179809220 shutdown_connections 2024-04-20T21:31:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.053+0000 7f2339a00700 1 -- 172.21.15.7:0/3179809220 wait complete. 2024-04-20T21:31:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.054+0000 7f2339a00700 1 Processor -- start 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 -- start start 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 0x7f233419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23341192d0 con 0x7f233410f040 2024-04-20T21:31:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2334119150 con 0x7f2334108f90 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2334119450 con 0x7f2334109950 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2333e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2333e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:51532/0 (socket says 172.21.15.7:51532) 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2339000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33636/0 (socket says 172.21.15.7:33636) 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2333e00700 1 -- 172.21.15.7:0/278256827 learned_addr learned my addr 172.21.15.7:0/278256827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.055+0000 7f2333400700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 0x7f233419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 -- 172.21.15.7:0/278256827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 msgr2=0x7f233419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 0x7f233419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 -- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 msgr2=0x7f23341a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 -- 172.21.15.7:0/278256827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23341a46b0 con 0x7f2334108f90 2024-04-20T21:31:11.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2339000700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2333e00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f232400e9a0 tx=0x7f232400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400ca10 con 0x7f2334108f90 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.056+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23341a49a0 con 0x7f2334108f90 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.057+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232400cbb0 con 0x7f2334108f90 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.057+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324018680 con 0x7f2334108f90 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.057+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23341ac220 con 0x7f2334108f90 2024-04-20T21:31:11.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.058+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2334069740 con 0x7f2334108f90 2024-04-20T21:31:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.060+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f2324010430 con 0x7f2334108f90 2024-04-20T21:31:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.062+0000 7f2330c00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 0x7f23180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.062+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f23240d06e0 con 0x7f2334108f90 2024-04-20T21:31:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.062+0000 7f2333400700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 0x7f23180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.062+0000 7f2333400700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 0x7f23180a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f233410e7a0 tx=0x7f232000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.066+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f232409d010 con 0x7f2334108f90 2024-04-20T21:31:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.361+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f23341a0a00 con 0x7f2334108f90 2024-04-20T21:31:11.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.367+0000 7f2330c00700 1 -- 172.21.15.7:0/278256827 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264988 (secure 0 0 0) 0x7f232409c910 con 0x7f2334108f90 2024-04-20T21:31:11.373 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 msgr2=0x7f23180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 0x7f23180a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f233410e7a0 tx=0x7f232000b260 comp rx=0 tx=0).stop 2024-04-20T21:31:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 msgr2=0x7f233419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f232400e9a0 tx=0x7f232400ecb0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 shutdown_connections 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f233410f040 0x7f23341a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334108f90 0x7f233419f6a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f23180a3cd0 0x7f23180a6190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 --2- 172.21.15.7:0/278256827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2334109950 0x7f233419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 >> 172.21.15.7:0/278256827 conn(0x7f23340787e0 msgr2=0x7f23340799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 shutdown_connections 2024-04-20T21:31:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:11.375+0000 7f2339a00700 1 -- 172.21.15.7:0/278256827 wait complete. 2024-04-20T21:31:12.170 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:12 smithi007 ceph-mon[16867]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-20T21:31:12.170 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:12 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/278256827' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:12.243 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2375060083},{"type":"v1","addr":"172.21.15.7:6801","nonce":2375060083}]},"active_addr":"172.21.15.7:6801/2375060083","active_change":"2024-04-20T21:22:48.211692+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.7:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1453786296}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3767646163}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1278013541}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":345179439}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3366257837}]}]} 2024-04-20T21:31:12.249 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-20T21:31:12.249 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-20T21:31:12.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:31:12.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:12 smithi016 ceph-mon[21409]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-20T21:31:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:12 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/278256827' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:12.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:12 smithi186 ceph-mon[22157]: pgmap v327: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-20T21:31:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:12 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/278256827' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-20T21:31:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:14.405 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:14 smithi007 ceph-mon[16867]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:14.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:14 smithi016 ceph-mon[21409]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:14.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:14 smithi186 ceph-mon[22157]: pgmap v328: 97 pgs: 97 active+clean; 628 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 -- 172.21.15.7:0/439224621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf54108a40 msgr2=0x7fbf54108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 --2- 172.21.15.7:0/439224621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf54108a40 0x7fbf54108e20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009990 tx=0x7fbf44009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 -- 172.21.15.7:0/439224621 shutdown_connections 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 --2- 172.21.15.7:0/439224621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406b100 0x7fbf5410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 --2- 172.21.15.7:0/439224621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf54108a40 0x7fbf54108e20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 --2- 172.21.15.7:0/439224621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf5406a760 0x7fbf5406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.919+0000 7fbf59800700 1 -- 172.21.15.7:0/439224621 >> 172.21.15.7:0/439224621 conn(0x7fbf54078900 msgr2=0x7fbf54078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.920+0000 7fbf59800700 1 -- 172.21.15.7:0/439224621 shutdown_connections 2024-04-20T21:31:14.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.920+0000 7fbf59800700 1 -- 172.21.15.7:0/439224621 wait complete. 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 Processor -- start 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 -- start start 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406b100 0x7fbf54101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf54108a40 0x7fbf541024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112d10 con 0x7fbf5406a760 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112b90 con 0x7fbf5406b100 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.921+0000 7fbf59800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbf54112e90 con 0x7fbf54108a40 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51760/0 (socket says 172.21.15.7:51760) 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 -- 172.21.15.7:0/1097559654 learned_addr learned my addr 172.21.15.7:0/1097559654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf4be00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406b100 0x7fbf54101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf53400700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf54108a40 0x7fbf541024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 -- 172.21.15.7:0/1097559654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf54108a40 msgr2=0x7fbf541024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf54108a40 0x7fbf541024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 -- 172.21.15.7:0/1097559654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406b100 msgr2=0x7fbf54101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406b100 0x7fbf54101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.922+0000 7fbf52a00700 1 -- 172.21.15.7:0/1097559654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a7710 con 0x7fbf5406a760 2024-04-20T21:31:14.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf52a00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000c00 tx=0x7fbf4400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44027070 con 0x7fbf5406a760 2024-04-20T21:31:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a79a0 con 0x7fbf5406a760 2024-04-20T21:31:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44003d70 con 0x7fbf5406a760 2024-04-20T21:31:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400f460 con 0x7fbf5406a760 2024-04-20T21:31:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.923+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a7e50 con 0x7fbf5406a760 2024-04-20T21:31:14.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.924+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf54069740 con 0x7fbf5406a760 2024-04-20T21:31:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.926+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fbf4402a070 con 0x7fbf5406a760 2024-04-20T21:31:14.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.928+0000 7fbf4b400700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 0x7fbf380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.928+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbf440d1230 con 0x7fbf5406a760 2024-04-20T21:31:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.928+0000 7fbf4be00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 0x7fbf380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.929+0000 7fbf4be00700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 0x7fbf380a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf541016c0 tx=0x7fbf3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:14.933+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbf44013070 con 0x7fbf5406a760 2024-04-20T21:31:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.170+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf5402df90 con 0x7fbf5406a760 2024-04-20T21:31:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.174+0000 7fbf4b400700 1 -- 172.21.15.7:0/1097559654 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22103 (secure 0 0 0) 0x7fbf4409dbf0 con 0x7fbf5406a760 2024-04-20T21:31:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:15.174 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:30:53.904734+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 msgr2=0x7fbf380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 0x7fbf380a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf541016c0 tx=0x7fbf3c009260 comp rx=0 tx=0).stop 2024-04-20T21:31:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 msgr2=0x7fbf54101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf44000c00 tx=0x7fbf4400bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 shutdown_connections 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fbf380a3cd0 0x7fbf380a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf54108a40 0x7fbf541024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf5406a760 0x7fbf54101a20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 --2- 172.21.15.7:0/1097559654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf5406b100 0x7fbf54101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 >> 172.21.15.7:0/1097559654 conn(0x7fbf54078900 msgr2=0x7fbf540ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 shutdown_connections 2024-04-20T21:31:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:15.178+0000 7fbf59800700 1 -- 172.21.15.7:0/1097559654 wait complete. 2024-04-20T21:31:15.910 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-20T21:31:15.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-20T21:31:16.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:16 smithi007 ceph-mon[16867]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:16 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1097559654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:16.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:16 smithi016 ceph-mon[21409]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:16.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:16 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1097559654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:16.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:16 smithi186 ceph-mon[22157]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-04-20T21:31:16.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:16 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1097559654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:18.306 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:18 smithi007 ceph-mon[16867]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-20T21:31:18.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:18 smithi016 ceph-mon[21409]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-20T21:31:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:18 smithi186 ceph-mon[22157]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-20T21:31:18.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 -- 172.21.15.7:0/62869454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8076f80 msgr2=0x7efcb8077360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 --2- 172.21.15.7:0/62869454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8076f80 0x7efcb8077360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efca80098f0 tx=0x7efca8009c00 comp rx=0 tx=0).stop 2024-04-20T21:31:18.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 -- 172.21.15.7:0/62869454 shutdown_connections 2024-04-20T21:31:18.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 --2- 172.21.15.7:0/62869454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8113930 0x7efcb8115d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 --2- 172.21.15.7:0/62869454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8076f80 0x7efcb8077360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 --2- 172.21.15.7:0/62869454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81133f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 -- 172.21.15.7:0/62869454 >> 172.21.15.7:0/62869454 conn(0x7efcb8100780 msgr2=0x7efcb8102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 -- 172.21.15.7:0/62869454 shutdown_connections 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.593+0000 7efcbee00700 1 -- 172.21.15.7:0/62869454 wait complete. 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.594+0000 7efcbee00700 1 Processor -- start 2024-04-20T21:31:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.594+0000 7efcbee00700 1 -- start start 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 0x7efcb81a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8113930 0x7efcb819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8119410 con 0x7efcb80778a0 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8119290 con 0x7efcb8113930 2024-04-20T21:31:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbee00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8119590 con 0x7efcb8076f80 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcaf400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcaf400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51780/0 (socket says 172.21.15.7:51780) 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcb7e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 0x7efcb81a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcaf400700 1 -- 172.21.15.7:0/2423624344 learned_addr learned my addr 172.21.15.7:0/2423624344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.595+0000 7efcbce00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8113930 0x7efcb819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 -- 172.21.15.7:0/2423624344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8113930 msgr2=0x7efcb819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8113930 0x7efcb819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 -- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 msgr2=0x7efcb81a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 -- 172.21.15.7:0/2423624344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb819ff70 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb7e00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 0x7efcb81a55c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efca800c010 tx=0x7efca800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8005180 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a0200 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca800b560 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8003a80 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.596+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81ac370 con 0x7efcb8076f80 2024-04-20T21:31:18.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.598+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb80786c0 con 0x7efcb8076f80 2024-04-20T21:31:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.600+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7efca800b710 con 0x7efcb8076f80 2024-04-20T21:31:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.601+0000 7efcb5600700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 0x7efc9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.601+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efca80d1390 con 0x7efcb8076f80 2024-04-20T21:31:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.601+0000 7efcaf400700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 0x7efc9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.602+0000 7efcaf400700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 0x7efc9c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb81a0e50 tx=0x7efca000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:18.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.605+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7efca8013030 con 0x7efcb8076f80 2024-04-20T21:31:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.839+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcb802df90 con 0x7efcb8076f80 2024-04-20T21:31:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.841+0000 7efcb5600700 1 -- 172.21.15.7:0/2423624344 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22103 (secure 0 0 0) 0x7efca809dd50 con 0x7efcb8076f80 2024-04-20T21:31:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:31:18.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:30:53.904734+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 msgr2=0x7efc9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 0x7efc9c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcb81a0e50 tx=0x7efca000b040 comp rx=0 tx=0).stop 2024-04-20T21:31:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 msgr2=0x7efcb81a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 0x7efcb81a55c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efca800c010 tx=0x7efca800fa70 comp rx=0 tx=0).stop 2024-04-20T21:31:18.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 shutdown_connections 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7efc9c0a3cd0 0x7efc9c0a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efcb8076f80 0x7efcb81a55c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8113930 0x7efcb819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 --2- 172.21.15.7:0/2423624344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcb80778a0 0x7efcb81a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.844+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 >> 172.21.15.7:0/2423624344 conn(0x7efcb8100780 msgr2=0x7efcb8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.845+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 shutdown_connections 2024-04-20T21:31:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:18.845+0000 7efcbee00700 1 -- 172.21.15.7:0/2423624344 wait complete. 2024-04-20T21:31:19.222 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:19 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2423624344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:19 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2423624344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:19 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2423624344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:31:19.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-20T21:31:19.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-20T21:31:19.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-20T21:31:19.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-20T21:31:19.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-20T21:31:19.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-20T21:31:19.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-20T21:31:19.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-20T21:31:19.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-20T21:31:19.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-20T21:31:19.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-20T21:31:19.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-20T21:31:20.259 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:20 smithi007 ceph-mon[16867]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-20T21:31:20.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:20 smithi016 ceph-mon[21409]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-20T21:31:20.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:20 smithi186 ceph-mon[22157]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-20T21:31:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:20.832 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.784 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:21.789 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:22.172 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:22 smithi007 ceph-mon[16867]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-20T21:31:22.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:22 smithi016 ceph-mon[21409]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-20T21:31:22.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:22 smithi186 ceph-mon[22157]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-20T21:31:24.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:24 smithi007 ceph-mon[16867]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-20T21:31:24.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:24 smithi016 ceph-mon[21409]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-20T21:31:24.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:24 smithi186 ceph-mon[22157]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-20T21:31:26.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:26 smithi007 ceph-mon[16867]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-20T21:31:26.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:26 smithi016 ceph-mon[21409]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-20T21:31:26.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:26 smithi186 ceph-mon[22157]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-20T21:31:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:28.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:28 smithi016 ceph-mon[21409]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-20T21:31:28.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:28 smithi186 ceph-mon[22157]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-20T21:31:28.478 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:28 smithi007 ceph-mon[16867]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 -- 172.21.15.7:0/3948258417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0755a0 msgr2=0x7f5bbc1136e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 --2- 172.21.15.7:0/3948258417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1136e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009990 tx=0x7f5ba4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 -- 172.21.15.7:0/3948258417 shutdown_connections 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 --2- 172.21.15.7:0/3948258417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc113c20 0x7f5bbc116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 --2- 172.21.15.7:0/3948258417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc074c80 0x7f5bbc075060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 --2- 172.21.15.7:0/3948258417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1136e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 -- 172.21.15.7:0/3948258417 >> 172.21.15.7:0/3948258417 conn(0x7f5bbc079bb0 msgr2=0x7f5bbc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 -- 172.21.15.7:0/3948258417 shutdown_connections 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.185+0000 7f5bc2600700 1 -- 172.21.15.7:0/3948258417 wait complete. 2024-04-20T21:31:29.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.186+0000 7f5bc2600700 1 Processor -- start 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.186+0000 7f5bc2600700 1 -- start start 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc074c80 0x7f5bbc1a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc113c20 0x7f5bbc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc1193b0 con 0x7f5bbc0755a0 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc119230 con 0x7f5bbc113c20 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bc2600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc119530 con 0x7f5bbc074c80 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bb2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bb2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54618/0 (socket says 172.21.15.7:54618) 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bb2a00700 1 -- 172.21.15.7:0/2045385886 learned_addr learned my addr 172.21.15.7:0/2045385886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc074c80 0x7f5bbc1a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbbe00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc113c20 0x7f5bbc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 -- 172.21.15.7:0/2045385886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc113c20 msgr2=0x7f5bbc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc113c20 0x7f5bbc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 -- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 msgr2=0x7f5bbc1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbb400700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc19ff90 con 0x7f5bbc074c80 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bbbe00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bbc113c20 0x7f5bbc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.187+0000 7f5bb2a00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bbc0755a0 0x7f5bbc1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bbb400700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc074c80 0x7f5bbc1a55e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9f0 tx=0x7f5bac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00cbd0 con 0x7f5bbc074c80 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bac00cd70 con 0x7f5bbc074c80 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018710 con 0x7f5bbc074c80 2024-04-20T21:31:29.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc1a0280 con 0x7f5bbc074c80 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.188+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1ac560 con 0x7f5bbc074c80 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.189+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5bbc113400 con 0x7f5bbc074c80 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.191+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f5bac010430 con 0x7f5bbc074c80 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb8c00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5ba00a3d50 0x7f5ba00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5bac0d0e10 con 0x7f5bbc074c80 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb2a00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5ba00a3d50 0x7f5ba00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb8c00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] conn(0x7f5ba00ad530 0x7f5ba00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bac004500 con 0x7f5ba00ad530 2024-04-20T21:31:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.194+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5bac0d11c0 con 0x7f5bbc074c80 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.195+0000 7f5bb2a00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5ba00a3d50 0x7f5ba00a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009750 tx=0x7f5ba400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.195+0000 7f5bbbe00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] conn(0x7f5ba00ad530 0x7f5ba00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.195+0000 7f5bbbe00700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] conn(0x7f5ba00ad530 0x7f5ba00af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.196+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== osd.2 v2:172.21.15.186:6800/1038897606 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bac004500 con 0x7f5ba00ad530 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.215+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 --> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5bbc076370 con 0x7f5ba00ad530 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.215+0000 7f5bb8c00700 1 -- 172.21.15.7:0/2045385886 <== osd.2 v2:172.21.15.186:6800/1038897606 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5bbc076370 con 0x7f5ba00ad530 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] conn(0x7f5ba00ad530 msgr2=0x7f5ba00af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] conn(0x7f5ba00ad530 0x7f5ba00af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5ba00a3d50 msgr2=0x7f5ba00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f5ba00a3d50 0x7f5ba00a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009750 tx=0x7f5ba400b280 comp rx=0 tx=0).stop 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc074c80 msgr2=0x7f5bbc1a55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 --2- 172.21.15.7:0/2045385886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5bbc074c80 0x7f5bbc1a55e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9f0 tx=0x7f5bac00ed00 comp rx=0 tx=0).stop 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bbbe00700 1 -- 172.21.15.7:0/2045385886 reap_dead start 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 shutdown_connections 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 >> 172.21.15.7:0/2045385886 conn(0x7f5bbc079bb0 msgr2=0x7f5bbc1129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 shutdown_connections 2024-04-20T21:31:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.216+0000 7f5bc2600700 1 -- 172.21.15.7:0/2045385886 wait complete. 2024-04-20T21:31:30.175 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:30 smithi007 ceph-mon[16867]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-20T21:31:30.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:30 smithi016 ceph-mon[21409]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-20T21:31:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:30 smithi186 ceph-mon[22157]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-20T21:31:31.188 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 -- 172.21.15.7:0/3575654137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 msgr2=0x7f52ec111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 --2- 172.21.15.7:0/3575654137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec111b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b0d0 tx=0x7f52e400b3e0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 -- 172.21.15.7:0/3575654137 shutdown_connections 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 --2- 172.21.15.7:0/3575654137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec111b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 --2- 172.21.15.7:0/3575654137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec104ea0 0x7f52ec105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 --2- 172.21.15.7:0/3575654137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec105850 0x7f52ec1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.295+0000 7f52f3200700 1 -- 172.21.15.7:0/3575654137 >> 172.21.15.7:0/3575654137 conn(0x7f52ec100730 msgr2=0x7f52ec102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.296+0000 7f52f3200700 1 -- 172.21.15.7:0/3575654137 shutdown_connections 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.296+0000 7f52f3200700 1 -- 172.21.15.7:0/3575654137 wait complete. 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 Processor -- start 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 -- start start 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec104ea0 0x7f52ec19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec105850 0x7f52ec19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec114fc0 con 0x7f52ec109f80 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec114e40 con 0x7f52ec104ea0 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f52ec115140 con 0x7f52ec105850 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f1200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54648/0 (socket says 172.21.15.7:54648) 2024-04-20T21:31:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.297+0000 7f52f1200700 1 -- 172.21.15.7:0/815646725 learned_addr learned my addr 172.21.15.7:0/815646725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52ebe00700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec104ea0 0x7f52ec19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52eb400700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec105850 0x7f52ec19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 -- 172.21.15.7:0/815646725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec105850 msgr2=0x7f52ec19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f52ec105850 0x7f52ec19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 -- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec104ea0 msgr2=0x7f52ec19b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52ec104ea0 0x7f52ec19b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ec1a0410 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52f1200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec19fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52e400d010 tx=0x7f52e40079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.298+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4003c00 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.299+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e4022070 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.299+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ec1a06a0 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.299+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e401b6c0 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.299+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ec1a7f80 con 0x7f52ec109f80 2024-04-20T21:31:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.300+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52ec063120 con 0x7f52ec109f80 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.302+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f52e4009ba0 con 0x7f52ec109f80 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.303+0000 7f52e8c00700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52d00a3cd0 0x7f52d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.303+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f52e400f020 con 0x7f52ec109f80 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.303+0000 7f52ebe00700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52d00a3cd0 0x7f52d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.304+0000 7f52e8c00700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] conn(0x7f52d00ad4b0 0x7f52d00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.304+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52d00aff80 con 0x7f52d00ad4b0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.304+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f52e40d35b0 con 0x7f52ec109f80 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.304+0000 7f52ebe00700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52d00a3cd0 0x7f52d00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009680 tx=0x7f52dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.304+0000 7f52eb400700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] conn(0x7f52d00ad4b0 0x7f52d00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.305+0000 7f52eb400700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] conn(0x7f52d00ad4b0 0x7f52d00af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.306+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== osd.1 v2:172.21.15.16:6800/3476654955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52d00aff80 con 0x7f52d00ad4b0 2024-04-20T21:31:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.325+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 --> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52ec0635a0 con 0x7f52d00ad4b0 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.325+0000 7f52e8c00700 1 -- 172.21.15.7:0/815646725 <== osd.1 v2:172.21.15.16:6800/3476654955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52ec0635a0 con 0x7f52d00ad4b0 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] conn(0x7f52d00ad4b0 msgr2=0x7f52d00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] conn(0x7f52d00ad4b0 0x7f52d00af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52d00a3cd0 msgr2=0x7f52d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f52d00a3cd0 0x7f52d00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009680 tx=0x7f52dc009260 comp rx=0 tx=0).stop 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 msgr2=0x7f52ec19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 --2- 172.21.15.7:0/815646725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52ec109f80 0x7f52ec19fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52e400d010 tx=0x7f52e40079a0 comp rx=0 tx=0).stop 2024-04-20T21:31:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f1200700 1 -- 172.21.15.7:0/815646725 reap_dead start 2024-04-20T21:31:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 shutdown_connections 2024-04-20T21:31:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.326+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 >> 172.21.15.7:0/815646725 conn(0x7f52ec100730 msgr2=0x7f52ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.327+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 shutdown_connections 2024-04-20T21:31:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:29.327+0000 7f52f3200700 1 -- 172.21.15.7:0/815646725 wait complete. 2024-04-20T21:31:32.179 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:32 smithi007 ceph-mon[16867]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:31:32.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:32 smithi016 ceph-mon[21409]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:31:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:32 smithi186 ceph-mon[22157]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:31:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 -- 172.21.15.7:0/3295453237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0108970 msgr2=0x7fd0c0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/3295453237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0108970 0x7fd0c0108d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009990 tx=0x7fd0b4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 -- 172.21.15.7:0/3295453237 shutdown_connections 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/3295453237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0103330 0x7fd0c010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/3295453237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0108970 0x7fd0c0108d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/3295453237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c0102970 0x7fd0c0102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.375+0000 7fd0c6a00700 1 -- 172.21.15.7:0/3295453237 >> 172.21.15.7:0/3295453237 conn(0x7fd0c00fe4a0 msgr2=0x7fd0c01008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.376+0000 7fd0c6a00700 1 -- 172.21.15.7:0/3295453237 shutdown_connections 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.376+0000 7fd0c6a00700 1 -- 172.21.15.7:0/3295453237 wait complete. 2024-04-20T21:31:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 Processor -- start 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 -- start start 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 0x7fd0c019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c0103330 0x7fd0c019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0108970 0x7fd0c019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112cd0 con 0x7fd0c0102970 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112b50 con 0x7fd0c0108970 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c6a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0112e50 con 0x7fd0c0103330 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 0x7fd0c019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c5600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 0x7fd0c019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54660/0 (socket says 172.21.15.7:54660) 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.377+0000 7fd0c5600700 1 -- 172.21.15.7:0/1262656748 learned_addr learned my addr 172.21.15.7:0/1262656748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c4c00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c0103330 0x7fd0c019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c6000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0108970 0x7fd0c019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 -- 172.21.15.7:0/1262656748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c0103330 msgr2=0x7fd0c019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd0c0103330 0x7fd0c019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 -- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0108970 msgr2=0x7fd0c019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0108970 0x7fd0c019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a0350 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c6000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0c0108970 0x7fd0c019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.378+0000 7fd0c5600700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 0x7fd0c019b340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4000c00 tx=0x7fd0b400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.379+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4027070 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.379+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0b4003d70 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.379+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b400f6c0 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.379+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a05e0 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.379+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a7ec0 con 0x7fd0c0102970 2024-04-20T21:31:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.380+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0c00ff420 con 0x7fd0c0102970 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.382+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fd0b402a070 con 0x7fd0c0102970 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.383+0000 7fd0be000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd0a80a3cd0 0x7fd0a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.383+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd0b40d1c30 con 0x7fd0c0102970 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0c4c00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd0a80a3cd0 0x7fd0a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0be000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] conn(0x7fd0a80ad4b0 0x7fd0a80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0b4009030 con 0x7fd0a80ad4b0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd0b40d1fe0 con 0x7fd0c0102970 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0c6000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] conn(0x7fd0a80ad4b0 0x7fd0a80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.384+0000 7fd0c4c00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd0a80a3cd0 0x7fd0a80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0c010fc80 tx=0x7fd0ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.385+0000 7fd0c6000700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] conn(0x7fd0a80ad4b0 0x7fd0a80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.386+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== osd.7 v2:172.21.15.16:6816/4148897395 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0b4009030 con 0x7fd0a80ad4b0 2024-04-20T21:31:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.404+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 --> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0c01040b0 con 0x7fd0a80ad4b0 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.404+0000 7fd0be000700 1 -- 172.21.15.7:0/1262656748 <== osd.7 v2:172.21.15.16:6816/4148897395 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd0c01040b0 con 0x7fd0a80ad4b0 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] conn(0x7fd0a80ad4b0 msgr2=0x7fd0a80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] conn(0x7fd0a80ad4b0 0x7fd0a80af8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd0a80a3cd0 msgr2=0x7fd0a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd0a80a3cd0 0x7fd0a80a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0c010fc80 tx=0x7fd0ac009260 comp rx=0 tx=0).stop 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 msgr2=0x7fd0c019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 --2- 172.21.15.7:0/1262656748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c0102970 0x7fd0c019b340 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4000c00 tx=0x7fd0b400bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6000700 1 -- 172.21.15.7:0/1262656748 reap_dead start 2024-04-20T21:31:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 shutdown_connections 2024-04-20T21:31:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.405+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 >> 172.21.15.7:0/1262656748 conn(0x7fd0c00fe4a0 msgr2=0x7fd0c00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.406+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 shutdown_connections 2024-04-20T21:31:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:32.406+0000 7fd0c6a00700 1 -- 172.21.15.7:0/1262656748 wait complete. 2024-04-20T21:31:33.929 INFO:teuthology.orchestra.run.smithi007.stdout:55834574927 2024-04-20T21:31:33.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-20T21:31:34.204 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:34 smithi007 ceph-mon[16867]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:34.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:34 smithi016 ceph-mon[21409]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:34 smithi186 ceph-mon[22157]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:36.072 INFO:teuthology.orchestra.run.smithi007.stdout:223338299436 2024-04-20T21:31:36.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-20T21:31:36.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:36 smithi007 ceph-mon[16867]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:36 smithi016 ceph-mon[21409]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:36 smithi186 ceph-mon[22157]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.582+0000 7ff3d0a00700 1 -- 172.21.15.7:0/3766517859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 msgr2=0x7ff3cc111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.582+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/3766517859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc111b60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400b0d0 tx=0x7ff3c400b3e0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 -- 172.21.15.7:0/3766517859 shutdown_connections 2024-04-20T21:31:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/3766517859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc111b60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/3766517859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 0x7ff3cc105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/3766517859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3cc105860 0x7ff3cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 -- 172.21.15.7:0/3766517859 >> 172.21.15.7:0/3766517859 conn(0x7ff3cc100760 msgr2=0x7ff3cc102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 -- 172.21.15.7:0/3766517859 shutdown_connections 2024-04-20T21:31:36.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.583+0000 7ff3d0a00700 1 -- 172.21.15.7:0/3766517859 wait complete. 2024-04-20T21:31:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 Processor -- start 2024-04-20T21:31:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 -- start start 2024-04-20T21:31:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 0x7ff3cc19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3cc105860 0x7ff3cc19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc114fd0 con 0x7ff3cc109f90 2024-04-20T21:31:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc114e50 con 0x7ff3cc104eb0 2024-04-20T21:31:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.584+0000 7ff3d0a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc115150 con 0x7ff3cc105860 2024-04-20T21:31:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41718/0 (socket says 172.21.15.7:41718) 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 -- 172.21.15.7:0/2982745512 learned_addr learned my addr 172.21.15.7:0/2982745512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3c9e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 0x7ff3cc19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3c9400700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3cc105860 0x7ff3cc19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 -- 172.21.15.7:0/2982745512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3cc105860 msgr2=0x7ff3cc19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3cc105860 0x7ff3cc19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 -- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 msgr2=0x7ff3cc19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 0x7ff3cc19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc1a03d0 con 0x7ff3cc109f90 2024-04-20T21:31:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3c9e00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc104eb0 0x7ff3cc19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.585+0000 7ff3ca800700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc19fc90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d010 tx=0x7ff3c40079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.586+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4003c00 con 0x7ff3cc109f90 2024-04-20T21:31:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.586+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc1a0660 con 0x7ff3cc109f90 2024-04-20T21:31:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.586+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4007e10 con 0x7ff3cc109f90 2024-04-20T21:31:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.586+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c401b8d0 con 0x7ff3cc109f90 2024-04-20T21:31:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.586+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a7f40 con 0x7ff3cc109f90 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.587+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3cc101bf0 con 0x7ff3cc109f90 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.589+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7ff3c4009ba0 con 0x7ff3cc109f90 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c2a00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff3b00a3d50 0x7ff3b00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff3c400f020 con 0x7ff3cc109f90 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c9e00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff3b00a3d50 0x7ff3b00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c2a00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] conn(0x7ff3b00ad530 0x7ff3b00af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3c402de50 con 0x7ff3b00ad530 2024-04-20T21:31:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.591+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff3c40d1a50 con 0x7ff3cc109f90 2024-04-20T21:31:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.592+0000 7ff3c9400700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] conn(0x7ff3b00ad530 0x7ff3b00af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.592+0000 7ff3c9e00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff3b00a3d50 0x7ff3b00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc011490 tx=0x7ff3bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.592+0000 7ff3c9400700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] conn(0x7ff3b00ad530 0x7ff3b00af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.593+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== osd.10 v2:172.21.15.16:6824/2592476634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3c402de50 con 0x7ff3b00ad530 2024-04-20T21:31:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.613+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 --> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3cc106390 con 0x7ff3b00ad530 2024-04-20T21:31:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3c2a00700 1 -- 172.21.15.7:0/2982745512 <== osd.10 v2:172.21.15.16:6824/2592476634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff3cc106390 con 0x7ff3b00ad530 2024-04-20T21:31:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] conn(0x7ff3b00ad530 msgr2=0x7ff3b00af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] conn(0x7ff3b00ad530 0x7ff3b00af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff3b00a3d50 msgr2=0x7ff3b00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff3b00a3d50 0x7ff3b00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc011490 tx=0x7ff3bc009260 comp rx=0 tx=0).stop 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 msgr2=0x7ff3cc19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3d0a00700 1 --2- 172.21.15.7:0/2982745512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3cc109f90 0x7ff3cc19fc90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400d010 tx=0x7ff3c40079a0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.614+0000 7ff3ca800700 1 -- 172.21.15.7:0/2982745512 reap_dead start 2024-04-20T21:31:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.615+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 shutdown_connections 2024-04-20T21:31:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.615+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 >> 172.21.15.7:0/2982745512 conn(0x7ff3cc100760 msgr2=0x7ff3cc1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.615+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 shutdown_connections 2024-04-20T21:31:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:36.615+0000 7ff3d0a00700 1 -- 172.21.15.7:0/2982745512 wait complete. 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 -- 172.21.15.7:0/2829331078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 msgr2=0x7f09fc108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 --2- 172.21.15.7:0/2829331078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 0x7f09fc108e40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0098f0 tx=0x7f09ec009c00 comp rx=0 tx=0).stop 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 -- 172.21.15.7:0/2829331078 shutdown_connections 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 --2- 172.21.15.7:0/2829331078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09fc103460 0x7f09fc10f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 --2- 172.21.15.7:0/2829331078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 0x7f09fc108e40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 --2- 172.21.15.7:0/2829331078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.757+0000 7f0a02c00700 1 -- 172.21.15.7:0/2829331078 >> 172.21.15.7:0/2829331078 conn(0x7f09fc079bb0 msgr2=0x7f09fc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.758+0000 7f0a02c00700 1 -- 172.21.15.7:0/2829331078 shutdown_connections 2024-04-20T21:31:37.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.758+0000 7f0a02c00700 1 -- 172.21.15.7:0/2829331078 wait complete. 2024-04-20T21:31:37.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.759+0000 7f0a02c00700 1 Processor -- start 2024-04-20T21:31:37.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.759+0000 7f0a02c00700 1 -- start start 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09fc103460 0x7f09fc19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 0x7f09fc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112d90 con 0x7f09fc102aa0 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112c10 con 0x7f09fc108a60 2024-04-20T21:31:37.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a02c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112f10 con 0x7f09fc103460 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f09fbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f09fbe00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41744/0 (socket says 172.21.15.7:41744) 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f09fbe00700 1 -- 172.21.15.7:0/4055129395 learned_addr learned my addr 172.21.15.7:0/4055129395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f0a00c00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 0x7f09fc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.760+0000 7f09f3e00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09fc103460 0x7f09fc19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 -- 172.21.15.7:0/4055129395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09fc103460 msgr2=0x7f09fc19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09fc103460 0x7f09fc19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 -- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 msgr2=0x7f09fc19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09fc108a60 0x7f09fc19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc1a0490 con 0x7f09fc102aa0 2024-04-20T21:31:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09fbe00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc19b480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005950 tx=0x7f09ec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec005230 con 0x7f09fc102aa0 2024-04-20T21:31:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f0a02c00700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc1a0720 con 0x7f09fc102aa0 2024-04-20T21:31:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ec00ba40 con 0x7f09fc102aa0 2024-04-20T21:31:37.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003b60 con 0x7f09fc102aa0 2024-04-20T21:31:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.761+0000 7f0a02c00700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a8000 con 0x7f09fc102aa0 2024-04-20T21:31:37.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.762+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09fc063160 con 0x7f09fc102aa0 2024-04-20T21:31:37.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.765+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f09ec003d00 con 0x7f09fc102aa0 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.766+0000 7f09f9600700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f09e00a3d50 0x7f09e00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.766+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f09ec0d18f0 con 0x7f09fc102aa0 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.766+0000 7f09f3e00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f09e00a3d50 0x7f09e00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.767+0000 7f09f9600700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] conn(0x7f09e00ad400 0x7f09e00af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.767+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09ec00b890 con 0x7f09e00ad400 2024-04-20T21:31:37.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.767+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f09ec0d1ca0 con 0x7f09fc102aa0 2024-04-20T21:31:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.767+0000 7f0a00c00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] conn(0x7f09e00ad400 0x7f09e00af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.767+0000 7f09f3e00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f09e00a3d50 0x7f09e00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009750 tx=0x7f09e4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:37.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.768+0000 7f0a00c00700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] conn(0x7f09e00ad400 0x7f09e00af820 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:37.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.769+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== osd.4 v2:172.21.15.16:6808/215460715 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09ec00b890 con 0x7f09e00ad400 2024-04-20T21:31:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.787+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 --> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09fc063560 con 0x7f09e00ad400 2024-04-20T21:31:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.788+0000 7f09f9600700 1 -- 172.21.15.7:0/4055129395 <== osd.4 v2:172.21.15.16:6808/215460715 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f09fc063560 con 0x7f09e00ad400 2024-04-20T21:31:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.788+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] conn(0x7f09e00ad400 msgr2=0x7f09e00af820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.788+0000 7f09f2000700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] conn(0x7f09e00ad400 0x7f09e00af820 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f09e00a3d50 msgr2=0x7f09e00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f09e00a3d50 0x7f09e00a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009750 tx=0x7f09e4009310 comp rx=0 tx=0).stop 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 msgr2=0x7f09fc19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 --2- 172.21.15.7:0/4055129395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09fc102aa0 0x7f09fc19b480 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09ec005950 tx=0x7f09ec005040 comp rx=0 tx=0).stop 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f0a00c00700 1 -- 172.21.15.7:0/4055129395 reap_dead start 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 shutdown_connections 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 >> 172.21.15.7:0/4055129395 conn(0x7f09fc079bb0 msgr2=0x7f09fc10df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 shutdown_connections 2024-04-20T21:31:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.789+0000 7f09f2000700 1 -- 172.21.15.7:0/4055129395 wait complete. 2024-04-20T21:31:38.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 -- 172.21.15.7:0/2618090204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c10e3c0 msgr2=0x7fb54c115f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 --2- 172.21.15.7:0/2618090204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c10e3c0 0x7fb54c115f90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b0d0 tx=0x7fb54400b3e0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 -- 172.21.15.7:0/2618090204 shutdown_connections 2024-04-20T21:31:38.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 --2- 172.21.15.7:0/2618090204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c10e3c0 0x7fb54c115f90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 --2- 172.21.15.7:0/2618090204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54c1092e0 0x7fb54c1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 --2- 172.21.15.7:0/2618090204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c109c90 0x7fb54c10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 -- 172.21.15.7:0/2618090204 >> 172.21.15.7:0/2618090204 conn(0x7fb54c076220 msgr2=0x7fb54c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 -- 172.21.15.7:0/2618090204 shutdown_connections 2024-04-20T21:31:38.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.090+0000 7fb553000700 1 -- 172.21.15.7:0/2618090204 wait complete. 2024-04-20T21:31:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.091+0000 7fb553000700 1 Processor -- start 2024-04-20T21:31:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.091+0000 7fb553000700 1 -- start start 2024-04-20T21:31:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c1092e0 0x7fb54c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54c109c90 0x7fb54c1a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c10e3c0 0x7fb54c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1193d0 con 0x7fb54c1092e0 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c119250 con 0x7fb54c109c90 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb553000700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c119550 con 0x7fb54c10e3c0 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb54be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c1092e0 0x7fb54c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb54be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c1092e0 0x7fb54c1a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41776/0 (socket says 172.21.15.7:41776) 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb54be00700 1 -- 172.21.15.7:0/1182416874 learned_addr learned my addr 172.21.15.7:0/1182416874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb54b400700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54c109c90 0x7fb54c1a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb551000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c10e3c0 0x7fb54c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb551000700 1 -- 172.21.15.7:0/1182416874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54c109c90 msgr2=0x7fb54c1a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb551000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54c109c90 0x7fb54c1a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb551000700 1 -- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c1092e0 msgr2=0x7fb54c1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.092+0000 7fb551000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c1092e0 0x7fb54c1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb551000700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c19fcf0 con 0x7fb54c10e3c0 2024-04-20T21:31:38.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb551000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c10e3c0 0x7fb54c19f590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b770 tx=0x7fb544007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544021020 con 0x7fb54c10e3c0 2024-04-20T21:31:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb544004c50 con 0x7fb54c10e3c0 2024-04-20T21:31:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c19ff80 con 0x7fb54c10e3c0 2024-04-20T21:31:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440052b0 con 0x7fb54c10e3c0 2024-04-20T21:31:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.093+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1ac2a0 con 0x7fb54c10e3c0 2024-04-20T21:31:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.094+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb54c077100 con 0x7fb54c10e3c0 2024-04-20T21:31:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.097+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7fb544024020 con 0x7fb54c10e3c0 2024-04-20T21:31:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb548c00700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb5300a3d50 0x7fb5300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb54400f020 con 0x7fb54c10e3c0 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb54be00700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb5300a3d50 0x7fb5300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb548c00700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] conn(0x7fb5300ad400 0x7fb5300af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5440020b0 con 0x7fb5300ad400 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.098+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb5440d1e10 con 0x7fb54c10e3c0 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.099+0000 7fb54b400700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] conn(0x7fb5300ad400 0x7fb5300af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.099+0000 7fb54be00700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb5300a3d50 0x7fb5300a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009d20 tx=0x7fb53c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.099+0000 7fb54b400700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] conn(0x7fb5300ad400 0x7fb5300af820 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.100+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== osd.0 v2:172.21.15.7:6802/2111694387 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5440020b0 con 0x7fb5300ad400 2024-04-20T21:31:38.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.120+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 --> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb54c10a7c0 con 0x7fb5300ad400 2024-04-20T21:31:38.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.120+0000 7fb548c00700 1 -- 172.21.15.7:0/1182416874 <== osd.0 v2:172.21.15.7:6802/2111694387 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb54c10a7c0 con 0x7fb5300ad400 2024-04-20T21:31:38.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] conn(0x7fb5300ad400 msgr2=0x7fb5300af820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] conn(0x7fb5300ad400 0x7fb5300af820 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb5300a3d50 msgr2=0x7fb5300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb5300a3d50 0x7fb5300a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009d20 tx=0x7fb53c009260 comp rx=0 tx=0).stop 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c10e3c0 msgr2=0x7fb54c19f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.121+0000 7fb553000700 1 --2- 172.21.15.7:0/1182416874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb54c10e3c0 0x7fb54c19f590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b770 tx=0x7fb544007a20 comp rx=0 tx=0).stop 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.122+0000 7fb551000700 1 -- 172.21.15.7:0/1182416874 reap_dead start 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.122+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 shutdown_connections 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.122+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 >> 172.21.15.7:0/1182416874 conn(0x7fb54c076220 msgr2=0x7fb54c0766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.122+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 shutdown_connections 2024-04-20T21:31:38.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:38.122+0000 7fb553000700 1 -- 172.21.15.7:0/1182416874 wait complete. 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 -- 172.21.15.7:0/1559578860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 msgr2=0x7f94ec108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 --2- 172.21.15.7:0/1559578860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 0x7f94ec108e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009990 tx=0x7f94dc009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 -- 172.21.15.7:0/1559578860 shutdown_connections 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 --2- 172.21.15.7:0/1559578860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06b100 0x7f94ec10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 --2- 172.21.15.7:0/1559578860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 0x7f94ec108e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 --2- 172.21.15.7:0/1559578860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec06a760 0x7f94ec06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 -- 172.21.15.7:0/1559578860 >> 172.21.15.7:0/1559578860 conn(0x7f94ec078900 msgr2=0x7f94ec078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 -- 172.21.15.7:0/1559578860 shutdown_connections 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.250+0000 7f94f2a00700 1 -- 172.21.15.7:0/1559578860 wait complete. 2024-04-20T21:31:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.251+0000 7f94f2a00700 1 Processor -- start 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 -- start start 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 0x7f94ec199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec06b100 0x7f94ec1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 0x7f94ec19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec112d30 con 0x7f94ec108a40 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec112bb0 con 0x7f94ec06b100 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.252+0000 7f94f2a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec112eb0 con 0x7f94ec06a760 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 0x7f94ec199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 0x7f94ec199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58502/0 (socket says 172.21.15.7:58502) 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 -- 172.21.15.7:0/3527581391 learned_addr learned my addr 172.21.15.7:0/3527581391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94eb400700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec06b100 0x7f94ec1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94f0a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 0x7f94ec19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 -- 172.21.15.7:0/3527581391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec06b100 msgr2=0x7f94ec1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec06b100 0x7f94ec1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 -- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 msgr2=0x7f94ec19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94ec108a40 0x7f94ec19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec19e220 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.253+0000 7f94ebe00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 0x7f94ec199260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00bae0 tx=0x7f94dc00bb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.254+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc026070 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.254+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc003d70 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.254+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec19e4b0 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.254+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc021410 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.254+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec101950 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.255+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94ec0631e0 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.257+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f94dc029070 con 0x7f94ec06a760 2024-04-20T21:31:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.258+0000 7f94e8c00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94d00a3cd0 0x7f94d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.258+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f94dc0d1a20 con 0x7f94ec06a760 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.258+0000 7f94eb400700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94d00a3cd0 0x7f94d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.259+0000 7f94e8c00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] conn(0x7f94d00ad4b0 0x7f94d00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.259+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94dc009030 con 0x7f94d00ad4b0 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.259+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f94dc0d1dd0 con 0x7f94ec06a760 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.259+0000 7f94f0a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] conn(0x7f94d00ad4b0 0x7f94d00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.259+0000 7f94eb400700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94d00a3cd0 0x7f94d00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94ec078fe0 tx=0x7f94d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.260+0000 7f94f0a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] conn(0x7f94d00ad4b0 0x7f94d00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.261+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== osd.8 v2:172.21.15.186:6816/3369879966 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94dc009030 con 0x7f94d00ad4b0 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.281+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 --> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94ec063660 con 0x7f94d00ad4b0 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94e8c00700 1 -- 172.21.15.7:0/3527581391 <== osd.8 v2:172.21.15.186:6816/3369879966 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94ec063660 con 0x7f94d00ad4b0 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] conn(0x7f94d00ad4b0 msgr2=0x7f94d00af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] conn(0x7f94d00ad4b0 0x7f94d00af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94d00a3cd0 msgr2=0x7f94d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94d00a3cd0 0x7f94d00a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94ec078fe0 tx=0x7f94d800b410 comp rx=0 tx=0).stop 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 msgr2=0x7f94ec199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f2a00700 1 --2- 172.21.15.7:0/3527581391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94ec06a760 0x7f94ec199260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00bae0 tx=0x7f94dc00bb10 comp rx=0 tx=0).stop 2024-04-20T21:31:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.282+0000 7f94f0a00700 1 -- 172.21.15.7:0/3527581391 reap_dead start 2024-04-20T21:31:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.283+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 shutdown_connections 2024-04-20T21:31:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.283+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 >> 172.21.15.7:0/3527581391 conn(0x7f94ec078900 msgr2=0x7f94ec0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.283+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 shutdown_connections 2024-04-20T21:31:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:35.283+0000 7f94f2a00700 1 -- 172.21.15.7:0/3527581391 wait complete. 2024-04-20T21:31:38.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:38 smithi007 ceph-mon[16867]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:38 smithi016 ceph-mon[21409]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:38.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:38 smithi186 ceph-mon[22157]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-20T21:31:38.884 INFO:teuthology.orchestra.run.smithi007.stdout:257698037800 2024-04-20T21:31:38.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 -- 172.21.15.7:0/3328774777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 msgr2=0x7f491410ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 --2- 172.21.15.7:0/3328774777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491410ec80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4904009990 tx=0x7f4904009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 -- 172.21.15.7:0/3328774777 shutdown_connections 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 --2- 172.21.15.7:0/3328774777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4914074e80 0x7f4914075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 --2- 172.21.15.7:0/3328774777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f4914111630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 --2- 172.21.15.7:0/3328774777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491410ec80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 -- 172.21.15.7:0/3328774777 >> 172.21.15.7:0/3328774777 conn(0x7f49140fe4d0 msgr2=0x7f49141008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 -- 172.21.15.7:0/3328774777 shutdown_connections 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.963+0000 7f491b200700 1 -- 172.21.15.7:0/3328774777 wait complete. 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.964+0000 7f491b200700 1 Processor -- start 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 -- start start 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4914074e80 0x7f491419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f491419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4914114e90 con 0x7f491410f1c0 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4914114d10 con 0x7f49140757a0 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491b200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4914115010 con 0x7f4914074e80 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491a800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f491419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491a800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f491419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41754/0 (socket says 172.21.15.7:41754) 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f4919400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f491a800700 1 -- 172.21.15.7:0/668533932 learned_addr learned my addr 172.21.15.7:0/668533932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.965+0000 7f4919e00700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4914074e80 0x7f491419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 -- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4914074e80 msgr2=0x7f491419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4914074e80 0x7f491419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 -- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 msgr2=0x7f491419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f491419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49141a0350 con 0x7f49140757a0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f491a800700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f491410f1c0 0x7f491419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.966+0000 7f4919400700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491419b880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49040059d0 tx=0x7f490400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.967+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904027070 con 0x7f49140757a0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.967+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49141a05e0 con 0x7f49140757a0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.967+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4904003d70 con 0x7f49140757a0 2024-04-20T21:31:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.967+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904022e30 con 0x7f49140757a0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.967+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141a7ec0 con 0x7f49140757a0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.968+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49140631e0 con 0x7f49140757a0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.972+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f490402a070 con 0x7f49140757a0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f490ea00700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f48fc0a3cd0 0x7f48fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f49040d1980 con 0x7f49140757a0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f4919e00700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f48fc0a3cd0 0x7f48fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f490ea00700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] conn(0x7f48fc0ad4b0 0x7f48fc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48fc0aff80 con 0x7f48fc0ad4b0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.974+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f49040d1e60 con 0x7f49140757a0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.975+0000 7f491a800700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] conn(0x7f48fc0ad4b0 0x7f48fc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.975+0000 7f4919e00700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f48fc0a3cd0 0x7f48fc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4908009c60 tx=0x7f4908009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.975+0000 7f491a800700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] conn(0x7f48fc0ad4b0 0x7f48fc0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.976+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== osd.5 v2:172.21.15.186:6808/955879969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48fc0aff80 con 0x7f48fc0ad4b0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.995+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 --> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49140635e0 con 0x7f48fc0ad4b0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.995+0000 7f490ea00700 1 -- 172.21.15.7:0/668533932 <== osd.5 v2:172.21.15.186:6808/955879969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49140635e0 con 0x7f48fc0ad4b0 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] conn(0x7f48fc0ad4b0 msgr2=0x7f48fc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] conn(0x7f48fc0ad4b0 0x7f48fc0af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f48fc0a3cd0 msgr2=0x7f48fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f48fc0a3cd0 0x7f48fc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4908009c60 tx=0x7f4908009260 comp rx=0 tx=0).stop 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 msgr2=0x7f491419b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491b200700 1 --2- 172.21.15.7:0/668533932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49140757a0 0x7f491419b880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49040059d0 tx=0x7f490400bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.996+0000 7f491a800700 1 -- 172.21.15.7:0/668533932 reap_dead start 2024-04-20T21:31:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.997+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 shutdown_connections 2024-04-20T21:31:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.997+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 >> 172.21.15.7:0/668533932 conn(0x7f49140fe4d0 msgr2=0x7f49140ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.997+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 shutdown_connections 2024-04-20T21:31:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:37.997+0000 7f491b200700 1 -- 172.21.15.7:0/668533932 wait complete. 2024-04-20T21:31:40.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:40 smithi007 ceph-mon[16867]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:40.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:40 smithi016 ceph-mon[21409]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:40.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:40 smithi186 ceph-mon[22157]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:31:40.593 INFO:teuthology.orchestra.run.smithi007.stdout:77309411403 2024-04-20T21:31:40.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-20T21:31:40.744 INFO:teuthology.orchestra.run.smithi007.stdout:167503724602 2024-04-20T21:31:40.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-20T21:31:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:313532612636 2024-04-20T21:31:41.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-20T21:31:41.985 INFO:teuthology.orchestra.run.smithi007.stdout:146028888127 2024-04-20T21:31:41.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-20T21:31:42.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:42 smithi007 ceph-mon[16867]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-20T21:31:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:42 smithi016 ceph-mon[21409]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-20T21:31:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:42 smithi186 ceph-mon[22157]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 -- 172.21.15.7:0/503383207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c1020e0 msgr2=0x7f755c10eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 --2- 172.21.15.7:0/503383207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c1020e0 0x7f755c10eda0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7548009990 tx=0x7f7548009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 -- 172.21.15.7:0/503383207 shutdown_connections 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 --2- 172.21.15.7:0/503383207 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 0x7f755c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 --2- 172.21.15.7:0/503383207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c10f2e0 0x7f755c1116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 --2- 172.21.15.7:0/503383207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c1020e0 0x7f755c10eda0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 -- 172.21.15.7:0/503383207 >> 172.21.15.7:0/503383207 conn(0x7f755c0fd370 msgr2=0x7f755c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 -- 172.21.15.7:0/503383207 shutdown_connections 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.356+0000 7f7562a00700 1 -- 172.21.15.7:0/503383207 wait complete. 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.358+0000 7f7562a00700 1 Processor -- start 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.358+0000 7f7562a00700 1 -- start start 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 0x7f755c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 0x7f755c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c10f2e0 0x7f755c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f755c114ee0 con 0x7f755c10f2e0 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f755c114d60 con 0x7f755c1020e0 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562a00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f755c115060 con 0x7f755c101730 2024-04-20T21:31:43.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 0x7f755c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7562000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c10f2e0 0x7f755c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7561600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 0x7f755c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 0x7f755c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60016/0 (socket says 172.21.15.7:60016) 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 -- 172.21.15.7:0/3724444457 learned_addr learned my addr 172.21.15.7:0/3724444457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 -- 172.21.15.7:0/3724444457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 msgr2=0x7f755c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 0x7f755c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 -- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c10f2e0 msgr2=0x7f755c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.359+0000 7f7560c00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c10f2e0 0x7f755c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7560c00700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755c1a0310 con 0x7f755c1020e0 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7561600700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f755c101730 0x7f755c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7562000700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f755c10f2e0 0x7f755c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7560c00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 0x7f755c19b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7548005910 tx=0x7f754800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548027070 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755c1a05a0 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548003d70 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800f460 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.360+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755c1a7e80 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.361+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f755c0631e0 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.364+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f754802a070 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.365+0000 7f755a000700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f75440a3cd0 0x7f75440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.366+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f75480d1a00 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.366+0000 7f755a000700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] conn(0x7f75440ad4b0 0x7f75440af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.366+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f754800b0b0 con 0x7f75440ad4b0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.366+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f75480d1db0 con 0x7f755c1020e0 2024-04-20T21:31:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.366+0000 7f7562000700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] conn(0x7f75440ad4b0 0x7f75440af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.367+0000 7f7561600700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f75440a3cd0 0x7f75440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.367+0000 7f7562000700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] conn(0x7f75440ad4b0 0x7f75440af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.368+0000 7f7561600700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f75440a3cd0 0x7f75440a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7550005950 tx=0x7f755000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.369+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== osd.9 v2:172.21.15.7:6826/3395112642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f754800b0b0 con 0x7f75440ad4b0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.393+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 --> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f755c0635e0 con 0x7f75440ad4b0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f755a000700 1 -- 172.21.15.7:0/3724444457 <== osd.9 v2:172.21.15.7:6826/3395112642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f755c0635e0 con 0x7f75440ad4b0 2024-04-20T21:31:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] conn(0x7f75440ad4b0 msgr2=0x7f75440af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] conn(0x7f75440ad4b0 0x7f75440af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f75440a3cd0 msgr2=0x7f75440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f75440a3cd0 0x7f75440a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7550005950 tx=0x7f755000a300 comp rx=0 tx=0).stop 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 msgr2=0x7f755c19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 --2- 172.21.15.7:0/3724444457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f755c1020e0 0x7f755c19b890 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7548005910 tx=0x7f754800bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562000700 1 -- 172.21.15.7:0/3724444457 reap_dead start 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 shutdown_connections 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 >> 172.21.15.7:0/3724444457 conn(0x7f755c0fd370 msgr2=0x7f755c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.394+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 shutdown_connections 2024-04-20T21:31:43.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:43.395+0000 7f7562a00700 1 -- 172.21.15.7:0/3724444457 wait complete. 2024-04-20T21:31:44.371 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:44 smithi007 ceph-mon[16867]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:31:44.437 INFO:teuthology.orchestra.run.smithi007.stdout:34359738455 2024-04-20T21:31:44.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-20T21:31:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:44 smithi016 ceph-mon[21409]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:31:44.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:44 smithi186 ceph-mon[22157]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.033+0000 7f86f6c00700 1 -- 172.21.15.7:0/701074316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 msgr2=0x7f86f01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.033+0000 7f86f6c00700 1 --2- 172.21.15.7:0/701074316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 0x7f86f01098d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f86d80099c0 tx=0x7f86d8009cd0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 -- 172.21.15.7:0/701074316 shutdown_connections 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 --2- 172.21.15.7:0/701074316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0109fb0 0x7f86f0111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 --2- 172.21.15.7:0/701074316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0104ed0 0x7f86f01052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 --2- 172.21.15.7:0/701074316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 0x7f86f01098d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 -- 172.21.15.7:0/701074316 >> 172.21.15.7:0/701074316 conn(0x7f86f0100760 msgr2=0x7f86f0102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 -- 172.21.15.7:0/701074316 shutdown_connections 2024-04-20T21:31:45.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.034+0000 7f86f6c00700 1 -- 172.21.15.7:0/701074316 wait complete. 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.035+0000 7f86f6c00700 1 Processor -- start 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.035+0000 7f86f6c00700 1 -- start start 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.035+0000 7f86f6c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0104ed0 0x7f86f019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.035+0000 7f86f6c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 0x7f86f019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f6c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 0x7f86f019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f6c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0114f90 con 0x7f86f0105880 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f6c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0114e10 con 0x7f86f0109fb0 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f6c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0115110 con 0x7f86f0104ed0 2024-04-20T21:31:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 0x7f86f019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86e7e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 0x7f86f019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 0x7f86f019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60034/0 (socket says 172.21.15.7:60034) 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 -- 172.21.15.7:0/4107345719 learned_addr learned my addr 172.21.15.7:0/4107345719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86efe00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0104ed0 0x7f86f019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 -- 172.21.15.7:0/4107345719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0104ed0 msgr2=0x7f86f019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0104ed0 0x7f86f019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 -- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 msgr2=0x7f86f019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86f0105880 0x7f86f019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86f4c00700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a0450 con 0x7f86f0109fb0 2024-04-20T21:31:45.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.036+0000 7f86efe00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f86f0104ed0 0x7f86f019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86f4c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 0x7f86f019fd10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86e800ef30 tx=0x7f86e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80092e0 con 0x7f86f0109fb0 2024-04-20T21:31:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e8004bb0 con 0x7f86f0109fb0 2024-04-20T21:31:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8004280 con 0x7f86f0109fb0 2024-04-20T21:31:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a06e0 con 0x7f86f0109fb0 2024-04-20T21:31:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.037+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a7fc0 con 0x7f86f0109fb0 2024-04-20T21:31:45.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.038+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f86f0063120 con 0x7f86f0109fb0 2024-04-20T21:31:45.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.041+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f86e801b030 con 0x7f86f0109fb0 2024-04-20T21:31:45.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.042+0000 7f86ed600700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f86d40a3d50 0x7f86d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.042+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f86e80d07a0 con 0x7f86f0109fb0 2024-04-20T21:31:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.042+0000 7f86efe00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f86d40a3d50 0x7f86d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.043+0000 7f86ed600700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] conn(0x7f86d40ad530 0x7f86d40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.043+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86d40b0000 con 0x7f86d40ad530 2024-04-20T21:31:45.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.043+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f86e80d0b50 con 0x7f86f0109fb0 2024-04-20T21:31:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.043+0000 7f86e7e00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] conn(0x7f86d40ad530 0x7f86d40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:45.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.043+0000 7f86efe00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f86d40a3d50 0x7f86d40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86e0011490 tx=0x7f86e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.044+0000 7f86e7e00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] conn(0x7f86d40ad530 0x7f86d40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:45.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.044+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== osd.6 v2:172.21.15.7:6818/2723605602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f86d40b0000 con 0x7f86d40ad530 2024-04-20T21:31:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.063+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 --> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f86f00635f0 con 0x7f86d40ad530 2024-04-20T21:31:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.063+0000 7f86ed600700 1 -- 172.21.15.7:0/4107345719 <== osd.6 v2:172.21.15.7:6818/2723605602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f86f00635f0 con 0x7f86d40ad530 2024-04-20T21:31:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] conn(0x7f86d40ad530 msgr2=0x7f86d40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] conn(0x7f86d40ad530 0x7f86d40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f86d40a3d50 msgr2=0x7f86d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f86d40a3d50 0x7f86d40a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f86e0011490 tx=0x7f86e0009260 comp rx=0 tx=0).stop 2024-04-20T21:31:45.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 msgr2=0x7f86f019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 --2- 172.21.15.7:0/4107345719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86f0109fb0 0x7f86f019fd10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f86e800ef30 tx=0x7f86e800c3a0 comp rx=0 tx=0).stop 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f4c00700 1 -- 172.21.15.7:0/4107345719 reap_dead start 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 shutdown_connections 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 >> 172.21.15.7:0/4107345719 conn(0x7f86f0100760 msgr2=0x7f86f01012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.064+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 shutdown_connections 2024-04-20T21:31:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:45.065+0000 7f86f6c00700 1 -- 172.21.15.7:0/4107345719 wait complete. 2024-04-20T21:31:45.912 INFO:teuthology.orchestra.run.smithi007.stdout:292057776163 2024-04-20T21:31:45.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-20T21:31:46.441 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:46 smithi007 ceph-mon[16867]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:46 smithi016 ceph-mon[21409]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:46 smithi186 ceph-mon[22157]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:46.945 INFO:teuthology.orchestra.run.smithi007.stdout:201863462964 2024-04-20T21:31:46.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-20T21:31:46.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 -- 172.21.15.7:0/1111566662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2828108a40 msgr2=0x7f2828108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 --2- 172.21.15.7:0/1111566662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2828108a40 0x7f2828108e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f281c009990 tx=0x7f281c009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 -- 172.21.15.7:0/1111566662 shutdown_connections 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 --2- 172.21.15.7:0/1111566662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2828108a40 0x7f2828108e20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 --2- 172.21.15.7:0/1111566662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282806b100 0x7f282810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 --2- 172.21.15.7:0/1111566662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282806a760 0x7f282806abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 -- 172.21.15.7:0/1111566662 >> 172.21.15.7:0/1111566662 conn(0x7f2828078900 msgr2=0x7f2828078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 -- 172.21.15.7:0/1111566662 shutdown_connections 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.621+0000 7f282fc00700 1 -- 172.21.15.7:0/1111566662 wait complete. 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.622+0000 7f282fc00700 1 Processor -- start 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.622+0000 7f282fc00700 1 -- start start 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.622+0000 7f282fc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282806a760 0x7f2828101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282fc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 0x7f2828101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282fc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2828108a40 0x7f28281024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282fc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2828112d70 con 0x7f282806a760 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282fc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2828112bf0 con 0x7f2828108a40 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282fc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2828112ef0 con 0x7f282806b100 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 0x7f2828101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282d200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282806a760 0x7f2828101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 0x7f2828101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55754/0 (socket says 172.21.15.7:55754) 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 -- 172.21.15.7:0/532867295 learned_addr learned my addr 172.21.15.7:0/532867295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f282dc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2828108a40 0x7f28281024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 -- 172.21.15.7:0/532867295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2828108a40 msgr2=0x7f28281024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2828108a40 0x7f28281024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 -- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282806a760 msgr2=0x7f2828101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282806a760 0x7f2828101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.623+0000 7f2827e00700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28281a7790 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f2827e00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 0x7f2828101f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9a0 tx=0x7f281400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281400c9c0 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f281400cb60 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28281a7a80 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2814018660 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.624+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28281a8010 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.625+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28280630e0 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.627+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f281400cd10 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f2825600700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3d50 0x7f28100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f28140d0a90 con 0x7f282806b100 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f282d200700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3d50 0x7f28100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f2825600700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] conn(0x7f28100ad530 0x7f28100af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2814016ec0 con 0x7f28100ad530 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.629+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f28140d0e40 con 0x7f282806b100 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.630+0000 7f282dc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] conn(0x7f28100ad530 0x7f28100af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.630+0000 7f282d200700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3d50 0x7f28100a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f281c006010 tx=0x7f281c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.630+0000 7f282dc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] conn(0x7f28100ad530 0x7f28100af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.631+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== osd.3 v2:172.21.15.7:6810/55132163 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2814016ec0 con 0x7f28100ad530 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.649+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 --> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2828063510 con 0x7f28100ad530 2024-04-20T21:31:46.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.650+0000 7f2825600700 1 -- 172.21.15.7:0/532867295 <== osd.3 v2:172.21.15.7:6810/55132163 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2828063510 con 0x7f28100ad530 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.650+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] conn(0x7f28100ad530 msgr2=0x7f28100af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.650+0000 7f282fc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] conn(0x7f28100ad530 0x7f28100af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3d50 msgr2=0x7f28100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3d50 0x7f28100a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f281c006010 tx=0x7f281c00b260 comp rx=0 tx=0).stop 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 msgr2=0x7f2828101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 --2- 172.21.15.7:0/532867295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282806b100 0x7f2828101f60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f281400e9a0 tx=0x7f281400ecb0 comp rx=0 tx=0).stop 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282dc00700 1 -- 172.21.15.7:0/532867295 reap_dead start 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 shutdown_connections 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 >> 172.21.15.7:0/532867295 conn(0x7f2828078900 msgr2=0x7f28280ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 shutdown_connections 2024-04-20T21:31:46.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.651+0000 7f282fc00700 1 -- 172.21.15.7:0/532867295 wait complete. 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 -- 172.21.15.7:0/369335331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 msgr2=0x7f19a4113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 --2- 172.21.15.7:0/369335331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a4113450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1990009990 tx=0x7f1990009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 -- 172.21.15.7:0/369335331 shutdown_connections 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 --2- 172.21.15.7:0/369335331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4113990 0x7f19a4115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 --2- 172.21.15.7:0/369335331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4074df0 0x7f19a40751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 --2- 172.21.15.7:0/369335331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a4113450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 -- 172.21.15.7:0/369335331 >> 172.21.15.7:0/369335331 conn(0x7f19a40fe4d0 msgr2=0x7f19a41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 -- 172.21.15.7:0/369335331 shutdown_connections 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.728+0000 7f19a3200700 1 -- 172.21.15.7:0/369335331 wait complete. 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.729+0000 7f19a3200700 1 Processor -- start 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 -- start start 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 0x7f19a419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4113990 0x7f19a41a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19a41192c0 con 0x7f19a4074df0 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4119140 con 0x7f19a4113990 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a3200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19a4119440 con 0x7f19a4075710 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 0x7f19a419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a1e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 0x7f19a419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39202/0 (socket says 172.21.15.7:39202) 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.730+0000 7f19a1e00700 1 -- 172.21.15.7:0/477194537 learned_addr learned my addr 172.21.15.7:0/477194537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a2800700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4113990 0x7f19a41a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1400700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 -- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 msgr2=0x7f19a419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 -- 172.21.15.7:0/477194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4113990 msgr2=0x7f19a41a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4113990 0x7f19a41a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a41a46e0 con 0x7f19a4074df0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a2800700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a4113990 0x7f19a41a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1400700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f19a4075710 0x7f19a419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.731+0000 7f19a1e00700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 0x7f19a419f6d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f199400eb30 tx=0x7f199400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.732+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199400cc90 con 0x7f19a4074df0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.732+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a41a49d0 con 0x7f19a4074df0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.732+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1994004500 con 0x7f19a4074df0 2024-04-20T21:31:47.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.732+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994010430 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.732+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a41ac250 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.733+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19a40631e0 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.735+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f1994003f70 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.736+0000 7f199aa00700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19880a3cd0 0x7f19880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.736+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f19940d0a90 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.736+0000 7f19a1400700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19880a3cd0 0x7f19880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.737+0000 7f199aa00700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] conn(0x7f19880ad4b0 0x7f19880af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.737+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f199400be50 con 0x7f19880ad4b0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.737+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f19940d0e40 con 0x7f19a4074df0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.737+0000 7f19a2800700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] conn(0x7f19880ad4b0 0x7f19880af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.737+0000 7f19a1400700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19880a3cd0 0x7f19880a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.738+0000 7f19a2800700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] conn(0x7f19880ad4b0 0x7f19880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:47.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.739+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== osd.11 v2:172.21.15.186:6824/2885587987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f199400be50 con 0x7f19880ad4b0 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.758+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 --> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19a40635e0 con 0x7f19880ad4b0 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.758+0000 7f199aa00700 1 -- 172.21.15.7:0/477194537 <== osd.11 v2:172.21.15.186:6824/2885587987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f19a40635e0 con 0x7f19880ad4b0 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] conn(0x7f19880ad4b0 msgr2=0x7f19880af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] conn(0x7f19880ad4b0 0x7f19880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19880a3cd0 msgr2=0x7f19880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19880a3cd0 0x7f19880a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990005fb0 comp rx=0 tx=0).stop 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 msgr2=0x7f19a419f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 --2- 172.21.15.7:0/477194537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19a4074df0 0x7f19a419f6d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f199400eb30 tx=0x7f199400ee40 comp rx=0 tx=0).stop 2024-04-20T21:31:47.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a2800700 1 -- 172.21.15.7:0/477194537 reap_dead start 2024-04-20T21:31:47.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 shutdown_connections 2024-04-20T21:31:47.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 >> 172.21.15.7:0/477194537 conn(0x7f19a40fe4d0 msgr2=0x7f19a40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:47.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 shutdown_connections 2024-04-20T21:31:47.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:46.759+0000 7f19a3200700 1 -- 172.21.15.7:0/477194537 wait complete. 2024-04-20T21:31:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.094 INFO:teuthology.orchestra.run.smithi007.stdout:111669149766 2024-04-20T21:31:48.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-20T21:31:48.232 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.234 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.235 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.237 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.246 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.246 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:48 smithi007 ceph-mon[16867]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:31:48.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:48 smithi016 ceph-mon[21409]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:31:48.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:48 smithi186 ceph-mon[22157]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:31:48.584 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:48.625 INFO:teuthology.orchestra.run.smithi007.stdout:347892351001 2024-04-20T21:31:48.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-20T21:31:49.161 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.161 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-20T21:31:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-20T21:31:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T21:31:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-20T21:31:49.162 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:49 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-20T21:31:49.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:49 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-20T21:31:49.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-04-20T21:31:49.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-20T21:31:49.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-20T21:31:49.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-20T21:31:49.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-20T21:31:49.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:49 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-20T21:31:50.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:50 smithi007 ceph-mon[16867]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:50.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-20T21:31:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:50 smithi016 ceph-mon[21409]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-20T21:31:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-20T21:31:50.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-20T21:31:50.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-20T21:31:50.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:50 smithi186 ceph-mon[22157]: osdmap e87: 12 total, 12 up, 12 in 2024-04-20T21:31:51.173 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:51 smithi007 ceph-mon[16867]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:51.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:51 smithi016 ceph-mon[21409]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:51.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:51 smithi186 ceph-mon[22157]: osdmap e88: 12 total, 12 up, 12 in 2024-04-20T21:31:52.409 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:52 smithi007 ceph-mon[16867]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:52.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:52 smithi016 ceph-mon[21409]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:52.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:52 smithi186 ceph-mon[22157]: pgmap v349: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:31:53.255 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:54.439 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:54 smithi016 ceph-mon[21409]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:54.439 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:54 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:54.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:54 smithi007 ceph-mon[16867]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:54.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:54 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:54 smithi186 ceph-mon[22157]: pgmap v350: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:54.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:54 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:31:55.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 -- 172.21.15.7:0/618166141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 msgr2=0x7f85d8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 --2- 172.21.15.7:0/618166141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d8108e70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009990 tx=0x7f85c8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 -- 172.21.15.7:0/618166141 shutdown_connections 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 --2- 172.21.15.7:0/618166141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8103460 0x7f85d810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 --2- 172.21.15.7:0/618166141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d8108e70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 --2- 172.21.15.7:0/618166141 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8102aa0 0x7f85d8102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 -- 172.21.15.7:0/618166141 >> 172.21.15.7:0/618166141 conn(0x7f85d8079c00 msgr2=0x7f85d807a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 -- 172.21.15.7:0/618166141 shutdown_connections 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.299+0000 7f85dca00700 1 -- 172.21.15.7:0/618166141 wait complete. 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.300+0000 7f85dca00700 1 Processor -- start 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.300+0000 7f85dca00700 1 -- start start 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 0x7f85d819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112de0 con 0x7f85d8108a90 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112c60 con 0x7f85d8102aa0 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85dca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8112f60 con 0x7f85d8103460 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35622/0 (socket says 172.21.15.7:35622) 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 -- 172.21.15.7:0/1731484951 learned_addr learned my addr 172.21.15.7:0/1731484951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5400700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 0x7f85d819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d6800700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 -- 172.21.15.7:0/1731484951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 msgr2=0x7f85d819b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 0x7f85d819b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 -- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 msgr2=0x7f85d819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5e00700 1 -- 172.21.15.7:0/1731484951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d81a0450 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d6800700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.301+0000 7f85d5400700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 0x7f85d819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85d5e00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8027070 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c8003d70 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d81a06e0 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c800f460 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.302+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a7fc0 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.303+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8069740 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.305+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f85c802a070 con 0x7f85d8102aa0 2024-04-20T21:31:55.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.306+0000 7f85cea00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 0x7f85bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:55.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.307+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f85c80d1250 con 0x7f85d8102aa0 2024-04-20T21:31:55.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.307+0000 7f85d5400700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 0x7f85bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:55.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.307+0000 7f85d5400700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 0x7f85bc0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85d807a500 tx=0x7f85c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:55.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.311+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f85c8013070 con 0x7f85d8102aa0 2024-04-20T21:31:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.560+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f85d81a8190 con 0x7f85d8102aa0 2024-04-20T21:31:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.561+0000 7f85cea00700 1 -- 172.21.15.7:0/1731484951 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f85c809db30 con 0x7f85d8102aa0 2024-04-20T21:31:55.561 INFO:teuthology.orchestra.run.smithi007.stdout:146028888131 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.564+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 msgr2=0x7f85bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 0x7f85bc0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f85d807a500 tx=0x7f85c4009260 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 msgr2=0x7f85d819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c800bab0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 shutdown_connections 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8108a90 0x7f85d819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8102aa0 0x7f85d819b440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f85bc0a3d50 0x7f85bc0a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 --2- 172.21.15.7:0/1731484951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f85d8103460 0x7f85d819b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:55.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 >> 172.21.15.7:0/1731484951 conn(0x7f85d8079c00 msgr2=0x7f85d810ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 shutdown_connections 2024-04-20T21:31:55.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:55.565+0000 7f85dca00700 1 -- 172.21.15.7:0/1731484951 wait complete. 2024-04-20T21:31:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:56.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-20T21:31:56.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:55 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1731484951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-20T21:31:56.245 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:55 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1731484951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:31:56.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: pgmap v351: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 13 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-20T21:31:56.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:55 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1731484951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-20T21:31:56.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 -- 172.21.15.7:0/1815434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848104ed0 msgr2=0x7f18481052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 --2- 172.21.15.7:0/1815434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848104ed0 0x7f18481052b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f183c0099c0 tx=0x7f183c009cd0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 -- 172.21.15.7:0/1815434223 shutdown_connections 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 --2- 172.21.15.7:0/1815434223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848109fb0 0x7f1848111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 --2- 172.21.15.7:0/1815434223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848104ed0 0x7f18481052b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 --2- 172.21.15.7:0/1815434223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848105880 0x7f18481098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 -- 172.21.15.7:0/1815434223 >> 172.21.15.7:0/1815434223 conn(0x7f1848100760 msgr2=0x7f1848102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 -- 172.21.15.7:0/1815434223 shutdown_connections 2024-04-20T21:31:56.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.526+0000 7f184f600700 1 -- 172.21.15.7:0/1815434223 wait complete. 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.527+0000 7f184f600700 1 Processor -- start 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 -- start start 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848105880 0x7f184819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848109fb0 0x7f184819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:56.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1848114ff0 con 0x7f1848109fb0 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1848114e70 con 0x7f1848105880 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184f600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1848115170 con 0x7f1848104ed0 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184d600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848109fb0 0x7f184819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184cc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:38068/0 (socket says 172.21.15.7:38068) 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.528+0000 7f184cc00700 1 -- 172.21.15.7:0/1036363074 learned_addr learned my addr 172.21.15.7:0/1036363074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:31:56.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f1847e00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848105880 0x7f184819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 -- 172.21.15.7:0/1036363074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848105880 msgr2=0x7f184819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848105880 0x7f184819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 -- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848109fb0 msgr2=0x7f184819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848109fb0 0x7f184819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 -- 172.21.15.7:0/1036363074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481a0430 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.529+0000 7f184cc00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f183c005b40 tx=0x7f183c0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.530+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c026070 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.530+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a06c0 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.530+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c003e40 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.530+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c00fcf0 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.530+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a7fa0 con 0x7f1848104ed0 2024-04-20T21:31:56.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.531+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1848069740 con 0x7f1848104ed0 2024-04-20T21:31:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.533+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f183c029070 con 0x7f1848104ed0 2024-04-20T21:31:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.535+0000 7f1845600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.535+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f183c0d1480 con 0x7f1848104ed0 2024-04-20T21:31:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.535+0000 7f1847e00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.536+0000 7f1847e00700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 0x7f18300a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f184810b510 tx=0x7f18340073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.537+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f183c030080 con 0x7f1848104ed0 2024-04-20T21:31:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:31:56.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.784+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f184802df90 con 0x7f1848104ed0 2024-04-20T21:31:56.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.785+0000 7f1845600700 1 -- 172.21.15.7:0/1036363074 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f183c013070 con 0x7f1848104ed0 2024-04-20T21:31:56.785 INFO:teuthology.orchestra.run.smithi007.stdout:257698037804 2024-04-20T21:31:56.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.788+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 msgr2=0x7f18300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.788+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 0x7f18300a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f184810b510 tx=0x7f18340073d0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.788+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 msgr2=0x7f184819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.788+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f183c005b40 tx=0x7f183c0038b0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 shutdown_connections 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1848104ed0 0x7f184819b420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1848109fb0 0x7f184819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 --2- 172.21.15.7:0/1036363074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1848105880 0x7f184819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 >> 172.21.15.7:0/1036363074 conn(0x7f1848100760 msgr2=0x7f1848102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 shutdown_connections 2024-04-20T21:31:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:31:56.789+0000 7f184f600700 1 -- 172.21.15.7:0/1036363074 wait complete. 2024-04-20T21:31:57.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:56 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1036363074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:31:57.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:56 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1036363074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:31:57.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:56 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1036363074' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-20T21:31:58.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:57 smithi186 ceph-mon[22157]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:57 smithi186 ceph-mon[22157]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-20T21:31:58.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:57 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:31:58.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:57 smithi007 ceph-mon[16867]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:57 smithi007 ceph-mon[16867]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-20T21:31:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:57 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:31:58.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:57 smithi016 ceph-mon[21409]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:31:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:57 smithi016 ceph-mon[21409]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-20T21:31:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:57 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:31:59.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888127 got 146028888131 for osd.4 2024-04-20T21:31:59.357 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:00.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:31:59 smithi007 ceph-mon[16867]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 24 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:00.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:31:59 smithi186 ceph-mon[22157]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 24 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:31:59 smithi016 ceph-mon[21409]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 24 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:00.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 -- 172.21.15.7:0/3418089416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c105880 msgr2=0x7f9a8c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 --2- 172.21.15.7:0/3418089416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c105880 0x7f9a8c1098d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9a780099c0 tx=0x7f9a78009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 -- 172.21.15.7:0/3418089416 shutdown_connections 2024-04-20T21:32:00.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 --2- 172.21.15.7:0/3418089416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 --2- 172.21.15.7:0/3418089416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c104ed0 0x7f9a8c1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 --2- 172.21.15.7:0/3418089416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c105880 0x7f9a8c1098d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.500+0000 7f9a92800700 1 -- 172.21.15.7:0/3418089416 >> 172.21.15.7:0/3418089416 conn(0x7f9a8c100760 msgr2=0x7f9a8c102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.501+0000 7f9a92800700 1 -- 172.21.15.7:0/3418089416 shutdown_connections 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.501+0000 7f9a92800700 1 -- 172.21.15.7:0/3418089416 wait complete. 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 Processor -- start 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 -- start start 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c104ed0 0x7f9a8c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c114fc0 con 0x7f9a8c109fb0 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c114e40 con 0x7f9a8c105880 2024-04-20T21:32:00.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.502+0000 7f9a92800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c115140 con 0x7f9a8c104ed0 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:35656/0 (socket says 172.21.15.7:35656) 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 -- 172.21.15.7:0/2579765654 learned_addr learned my addr 172.21.15.7:0/2579765654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8b400700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c104ed0 0x7f9a8c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8be00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 -- 172.21.15.7:0/2579765654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c104ed0 msgr2=0x7f9a8c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c104ed0 0x7f9a8c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 -- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 msgr2=0x7f9a8c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 -- 172.21.15.7:0/2579765654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c1a03a0 con 0x7f9a8c105880 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8be00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.503+0000 7f9a8aa00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a78003e00 tx=0x7f9a78003e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:00.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.504+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a78009e60 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.504+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a78003680 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.504+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a780173f0 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.504+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c1a0630 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.504+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1a7f10 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.505+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c1063b0 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.507+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9a7800f580 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.508+0000 7f9a83e00700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 0x7f9a700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.508+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9a780d1410 con 0x7f9a8c105880 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.509+0000 7f9a8b400700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 0x7f9a700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.509+0000 7f9a8b400700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 0x7f9a700a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009680 tx=0x7f9a7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:00.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.511+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9a78013070 con 0x7f9a8c105880 2024-04-20T21:32:00.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.749+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9a8c1a80e0 con 0x7f9a8c105880 2024-04-20T21:32:00.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.749+0000 7f9a83e00700 1 -- 172.21.15.7:0/2579765654 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a7809dd50 con 0x7f9a8c105880 2024-04-20T21:32:00.749 INFO:teuthology.orchestra.run.smithi007.stdout:55834574932 2024-04-20T21:32:00.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 msgr2=0x7f9a700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 0x7f9a700a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009680 tx=0x7f9a7c009260 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 msgr2=0x7f9a8c19b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a78003e00 tx=0x7f9a78003e30 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 shutdown_connections 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9a700a3cd0 0x7f9a700a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a8c104ed0 0x7f9a8c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a8c109fb0 0x7f9a8c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 --2- 172.21.15.7:0/2579765654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a8c105880 0x7f9a8c19b8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.752+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 >> 172.21.15.7:0/2579765654 conn(0x7f9a8c100760 msgr2=0x7f9a8c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.753+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 shutdown_connections 2024-04-20T21:32:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:00.753+0000 7f9a92800700 1 -- 172.21.15.7:0/2579765654 wait complete. 2024-04-20T21:32:01.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:00 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2579765654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:01.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:00 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2579765654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:01.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:00 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2579765654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.877+0000 7f8a5dc00700 1 -- 172.21.15.7:0/64835170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58075710 msgr2=0x7f8a58113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.877+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/64835170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58075710 0x7f8a58113450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009990 tx=0x7f8a44009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 -- 172.21.15.7:0/64835170 shutdown_connections 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/64835170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58074df0 0x7f8a580751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/64835170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58113990 0x7f8a58115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/64835170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58075710 0x7f8a58113450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 -- 172.21.15.7:0/64835170 >> 172.21.15.7:0/64835170 conn(0x7f8a580fe4d0 msgr2=0x7f8a581008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:01.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 -- 172.21.15.7:0/64835170 shutdown_connections 2024-04-20T21:32:01.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.878+0000 7f8a5dc00700 1 -- 172.21.15.7:0/64835170 wait complete. 2024-04-20T21:32:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.879+0000 7f8a5dc00700 1 Processor -- start 2024-04-20T21:32:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.879+0000 7f8a5dc00700 1 -- start start 2024-04-20T21:32:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:01.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075710 0x7f8a581a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58113990 0x7f8a5819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58119320 con 0x7f8a58074df0 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581191a0 con 0x7f8a58075710 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5dc00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581194a0 con 0x7f8a58113990 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a57e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a57e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47648/0 (socket says 172.21.15.7:47648) 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a57e00700 1 -- 172.21.15.7:0/343257412 learned_addr learned my addr 172.21.15.7:0/343257412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a5d200700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58113990 0x7f8a5819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.880+0000 7f8a57400700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075710 0x7f8a581a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:01.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 -- 172.21.15.7:0/343257412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58113990 msgr2=0x7f8a5819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58113990 0x7f8a5819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 -- 172.21.15.7:0/343257412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075710 msgr2=0x7f8a581a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075710 0x7f8a581a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:01.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 -- 172.21.15.7:0/343257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5819fd60 con 0x7f8a58074df0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a57e00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9c0 tx=0x7f8a4800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800cbd0 con 0x7f8a58074df0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4800cd70 con 0x7f8a58074df0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010430 con 0x7f8a58074df0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.881+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a581a0050 con 0x7f8a58074df0 2024-04-20T21:32:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.882+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a581ac310 con 0x7f8a58074df0 2024-04-20T21:32:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.883+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a58069740 con 0x7f8a58074df0 2024-04-20T21:32:01.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.885+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f8a48004750 con 0x7f8a58074df0 2024-04-20T21:32:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.886+0000 7f8a54c00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 0x7f8a3c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.886+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8a480d0af0 con 0x7f8a58074df0 2024-04-20T21:32:01.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.887+0000 7f8a57400700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 0x7f8a3c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:01.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.887+0000 7f8a57400700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 0x7f8a3c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b2e0 tx=0x7f8a4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:01.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:01.889+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8a4809d3d0 con 0x7f8a58074df0 2024-04-20T21:32:02.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.122+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8a581a0b30 con 0x7f8a58074df0 2024-04-20T21:32:02.122 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:01 smithi007 ceph-mon[16867]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:02.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.123+0000 7f8a54c00700 1 -- 172.21.15.7:0/343257412 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8a4809ccd0 con 0x7f8a58074df0 2024-04-20T21:32:02.123 INFO:teuthology.orchestra.run.smithi007.stdout:313532612641 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 msgr2=0x7f8a3c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 0x7f8a3c0a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b2e0 tx=0x7f8a4401a040 comp rx=0 tx=0).stop 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 msgr2=0x7f8a581a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e9c0 tx=0x7f8a4800ecd0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 shutdown_connections 2024-04-20T21:32:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f8a3c0a3d50 0x7f8a3c0a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8a58113990 0x7f8a5819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a58074df0 0x7f8a581a5440 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 --2- 172.21.15.7:0/343257412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a58075710 0x7f8a581a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.126+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 >> 172.21.15.7:0/343257412 conn(0x7f8a580fe4d0 msgr2=0x7f8a580ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.127+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 shutdown_connections 2024-04-20T21:32:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:02.127+0000 7f8a5dc00700 1 -- 172.21.15.7:0/343257412 wait complete. 2024-04-20T21:32:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:01 smithi016 ceph-mon[21409]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:02.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:01 smithi186 ceph-mon[22157]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 -- 172.21.15.7:0/1074085276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c1020e0 msgr2=0x7f087c10eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 --2- 172.21.15.7:0/1074085276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c1020e0 0x7f087c10eda0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0868009990 tx=0x7f0868009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 -- 172.21.15.7:0/1074085276 shutdown_connections 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 --2- 172.21.15.7:0/1074085276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c101730 0x7f087c101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 --2- 172.21.15.7:0/1074085276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c10f2e0 0x7f087c1116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 --2- 172.21.15.7:0/1074085276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c1020e0 0x7f087c10eda0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.102+0000 7f0881200700 1 -- 172.21.15.7:0/1074085276 >> 172.21.15.7:0/1074085276 conn(0x7f087c0fd370 msgr2=0x7f087c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.103+0000 7f0881200700 1 -- 172.21.15.7:0/1074085276 shutdown_connections 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.103+0000 7f0881200700 1 -- 172.21.15.7:0/1074085276 wait complete. 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 Processor -- start 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 -- start start 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c101730 0x7f087c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 0x7f087c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f087c114e90 con 0x7f087c1020e0 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f087c114d10 con 0x7f087c101730 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.104+0000 7f0881200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f087c115010 con 0x7f087c10f2e0 2024-04-20T21:32:03.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44202/0 (socket says 172.21.15.7:44202) 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 -- 172.21.15.7:0/1128954813 learned_addr learned my addr 172.21.15.7:0/1128954813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087be00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 0x7f087c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087b400700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c101730 0x7f087c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 -- 172.21.15.7:0/1128954813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 msgr2=0x7f087c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 0x7f087c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 -- 172.21.15.7:0/1128954813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c101730 msgr2=0x7f087c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c101730 0x7f087c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 -- 172.21.15.7:0/1128954813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087c1a0310 con 0x7f087c1020e0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087be00700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 0x7f087c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.105+0000 7f087aa00700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0868000c00 tx=0x7f086800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.106+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868027070 con 0x7f087c1020e0 2024-04-20T21:32:03.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.106+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087c1a05a0 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.106+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0868003d70 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.106+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086800f460 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.106+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087c1a7e80 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.107+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f087c069740 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.109+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f086802a070 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.111+0000 7f0873e00700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 0x7f08600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.111+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f08680d1290 con 0x7f087c1020e0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.111+0000 7f087b400700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 0x7f08600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.112+0000 7f087b400700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 0x7f08600a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f086c009680 tx=0x7f086c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:03.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.115+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f0868013070 con 0x7f087c1020e0 2024-04-20T21:32:03.217 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:02 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/343257412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.356+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f087c02df90 con 0x7f087c1020e0 2024-04-20T21:32:03.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.357+0000 7f0873e00700 1 -- 172.21.15.7:0/1128954813 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f086809db70 con 0x7f087c1020e0 2024-04-20T21:32:03.357 INFO:teuthology.orchestra.run.smithi007.stdout:77309411409 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.359+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 msgr2=0x7f08600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.359+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 0x7f08600a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f086c009680 tx=0x7f086c009260 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 msgr2=0x7f087c19b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0868000c00 tx=0x7f086800bab0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 shutdown_connections 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f08600a3d50 0x7f08600a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f087c10f2e0 0x7f087c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c101730 0x7f087c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 --2- 172.21.15.7:0/1128954813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f087c1020e0 0x7f087c19b840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 >> 172.21.15.7:0/1128954813 conn(0x7f087c0fd370 msgr2=0x7f087c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:03.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 shutdown_connections 2024-04-20T21:32:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.360+0000 7f0881200700 1 -- 172.21.15.7:0/1128954813 wait complete. 2024-04-20T21:32:03.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:02 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/343257412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:03.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:02 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/343257412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-20T21:32:04.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037804 for osd.8 2024-04-20T21:32:04.194 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:04.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:03 smithi007 ceph-mon[16867]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:04.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:03 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1128954813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:04.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:03 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1464047460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 -- 172.21.15.7:0/594834625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 msgr2=0x7f9ae410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 --2- 172.21.15.7:0/594834625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae410f930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b0d0 tx=0x7f9adc00b3e0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 -- 172.21.15.7:0/594834625 shutdown_connections 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 --2- 172.21.15.7:0/594834625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4107950 0x7f9ae4107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 --2- 172.21.15.7:0/594834625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae410f930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 --2- 172.21.15.7:0/594834625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4102b00 0x7f9ae4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.217+0000 7f9ae9800700 1 -- 172.21.15.7:0/594834625 >> 172.21.15.7:0/594834625 conn(0x7f9ae4079bb0 msgr2=0x7f9ae4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.218+0000 7f9ae9800700 1 -- 172.21.15.7:0/594834625 shutdown_connections 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.218+0000 7f9ae9800700 1 -- 172.21.15.7:0/594834625 wait complete. 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 Processor -- start 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 -- start start 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 0x7f9ae4074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 0x7f9ae40757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.219+0000 7f9ae9800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4112db0 con 0x7f9ae41034c0 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae9800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4112c30 con 0x7f9ae4107950 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4112f30 con 0x7f9ae4102b00 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44220/0 (socket says 172.21.15.7:44220) 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 -- 172.21.15.7:0/1464047460 learned_addr learned my addr 172.21.15.7:0/1464047460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2a00700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 0x7f9ae4074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae3400700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 0x7f9ae40757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 -- 172.21.15.7:0/1464047460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 msgr2=0x7f9ae4074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 0x7f9ae4074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 -- 172.21.15.7:0/1464047460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 msgr2=0x7f9ae40757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 0x7f9ae40757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2000700 1 -- 172.21.15.7:0/1464047460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae41a76d0 con 0x7f9ae41034c0 2024-04-20T21:32:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae2a00700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 0x7f9ae4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.220+0000 7f9ae3400700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 0x7f9ae40757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9ae2000700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000e9a0 tx=0x7f9ad000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000cbd0 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ad000cd70 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae41a79c0 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0018750 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.221+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae41a7f00 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.222+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae4069740 con 0x7f9ae41034c0 2024-04-20T21:32:04.290 INFO:teuthology.orchestra.run.smithi007.stdout:223338299442 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.225+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f9ad0010430 con 0x7f9ae41034c0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.226+0000 7f9adb400700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 0x7f9ac80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.226+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9ad00d0860 con 0x7f9ae41034c0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.226+0000 7f9ae2a00700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 0x7f9ac80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.227+0000 7f9ae2a00700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 0x7f9ac80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009fd0 tx=0x7f9ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.230+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9ad009d0b0 con 0x7f9ae41034c0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.471+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9ae402df90 con 0x7f9ae41034c0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.472+0000 7f9adb400700 1 -- 172.21.15.7:0/1464047460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9ad009c9b0 con 0x7f9ae41034c0 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 msgr2=0x7f9ac80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 0x7f9ac80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009fd0 tx=0x7f9ad4009260 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 msgr2=0x7f9ae4075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000e9a0 tx=0x7f9ad000ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 shutdown_connections 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ac80a3cd0 0x7f9ac80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9ae4102b00 0x7f9ae4074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae4107950 0x7f9ae40757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 --2- 172.21.15.7:0/1464047460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae41034c0 0x7f9ae4075290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 >> 172.21.15.7:0/1464047460 conn(0x7f9ae4079bb0 msgr2=0x7f9ae4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 shutdown_connections 2024-04-20T21:32:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:03.476+0000 7f9ae9800700 1 -- 172.21.15.7:0/1464047460 wait complete. 2024-04-20T21:32:04.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:03 smithi186 ceph-mon[22157]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:04.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:03 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1128954813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:04.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:03 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1464047460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:03 smithi016 ceph-mon[21409]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-04-20T21:32:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:03 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1128954813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-20T21:32:04.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:03 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1464047460' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-20T21:32:05.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299442 for osd.7 2024-04-20T21:32:05.849 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:06.095 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:05 smithi007 ceph-mon[16867]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:06.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:05 smithi016 ceph-mon[21409]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:05 smithi186 ceph-mon[22157]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:06.551 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574932 for osd.1 2024-04-20T21:32:06.551 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:07.135 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3063742193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:07.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.094+0000 7f31f9800700 1 -- 172.21.15.7:0/3254571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 msgr2=0x7f31f4111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.094+0000 7f31f9800700 1 --2- 172.21.15.7:0/3254571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f4111a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00b0a0 tx=0x7f31ec00b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 -- 172.21.15.7:0/3254571012 shutdown_connections 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 --2- 172.21.15.7:0/3254571012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 0x7f31f41051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 --2- 172.21.15.7:0/3254571012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f4111a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 --2- 172.21.15.7:0/3254571012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f41097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 -- 172.21.15.7:0/3254571012 >> 172.21.15.7:0/3254571012 conn(0x7f31f4100630 msgr2=0x7f31f4102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 -- 172.21.15.7:0/3254571012 shutdown_connections 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.095+0000 7f31f9800700 1 -- 172.21.15.7:0/3254571012 wait complete. 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.096+0000 7f31f9800700 1 Processor -- start 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.096+0000 7f31f9800700 1 -- start start 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 0x7f31f419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4114ea0 con 0x7f31f4105770 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4114d20 con 0x7f31f4109ea0 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4115020 con 0x7f31f4104dc0 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44230/0 (socket says 172.21.15.7:44230) 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f3400700 1 -- 172.21.15.7:0/3063742193 learned_addr learned my addr 172.21.15.7:0/3063742193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f3e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 0x7f31f419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 -- 172.21.15.7:0/3063742193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 msgr2=0x7f31f419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 0x7f31f419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 -- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 msgr2=0x7f31f419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f8e00700 1 -- 172.21.15.7:0/3063742193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41a02a0 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.097+0000 7f31f3400700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f8e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f419fb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009bc0 tx=0x7f31ec004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec021070 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ec007b80 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0052a0 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a0530 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.098+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a7e10 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.099+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4069740 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.101+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f31ec024070 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.103+0000 7f31f0c00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 0x7f31d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.103+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f31ec0d1400 con 0x7f31f4109ea0 2024-04-20T21:32:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.103+0000 7f31f3e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 0x7f31d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.104+0000 7f31f3e00700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 0x7f31d80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31e4004630 tx=0x7f31e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.107+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f31ec09dce0 con 0x7f31f4109ea0 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stdout:292057776167 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.346+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f31f41a7fe0 con 0x7f31f4109ea0 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.346+0000 7f31f0c00700 1 -- 172.21.15.7:0/3063742193 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f31ec09d5e0 con 0x7f31f4109ea0 2024-04-20T21:32:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.349+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 msgr2=0x7f31d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.349+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 0x7f31d80a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f31e4004630 tx=0x7f31e400b3c0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.349+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 msgr2=0x7f31f419fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.349+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f419fb60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009bc0 tx=0x7f31ec004d00 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 shutdown_connections 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f31d80a3d50 0x7f31d80a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f31f4104dc0 0x7f31f419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31f4109ea0 0x7f31f419fb60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 --2- 172.21.15.7:0/3063742193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31f4105770 0x7f31f419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 >> 172.21.15.7:0/3063742193 conn(0x7f31f4100630 msgr2=0x7f31f4101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 shutdown_connections 2024-04-20T21:32:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:06.350+0000 7f31f9800700 1 -- 172.21.15.7:0/3063742193 wait complete. 2024-04-20T21:32:07.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:07 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3063742193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:07.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:07 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3063742193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-20T21:32:08.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:08 smithi007 ceph-mon[16867]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:08.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/351851368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:08.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:08 smithi016 ceph-mon[21409]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:08.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/351851368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:08.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:08 smithi186 ceph-mon[22157]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-04-20T21:32:08.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/351851368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-20T21:32:08.560 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411409 for osd.2 2024-04-20T21:32:08.560 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:09.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612636 got 313532612641 for osd.10 2024-04-20T21:32:09.158 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:09.256 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 -- 172.21.15.7:0/151740372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21581092e0 msgr2=0x7f21581096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 --2- 172.21.15.7:0/151740372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21581092e0 0x7f21581096c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2148009920 tx=0x7f2148009c30 comp rx=0 tx=0).stop 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 -- 172.21.15.7:0/151740372 shutdown_connections 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 --2- 172.21.15.7:0/151740372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21581092e0 0x7f21581096c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 --2- 172.21.15.7:0/151740372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f215810e3c0 0x7f2158115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 --2- 172.21.15.7:0/151740372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2158109c90 0x7f215810dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 -- 172.21.15.7:0/151740372 >> 172.21.15.7:0/151740372 conn(0x7f2158076220 msgr2=0x7f2158107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 -- 172.21.15.7:0/151740372 shutdown_connections 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.686+0000 7f215ea00700 1 -- 172.21.15.7:0/151740372 wait complete. 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.687+0000 7f215ea00700 1 Processor -- start 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 -- start start 2024-04-20T21:32:09.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2158109c90 0x7f21581a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f215810e3c0 0x7f215819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2158119420 con 0x7f21581092e0 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21581192a0 con 0x7f2158109c90 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ea00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f21581195a0 con 0x7f215810e3c0 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f2157e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f2157e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44246/0 (socket says 172.21.15.7:44246) 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f2157e00700 1 -- 172.21.15.7:0/351851368 learned_addr learned my addr 172.21.15.7:0/351851368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f2157400700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2158109c90 0x7f21581a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.688+0000 7f215ca00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f215810e3c0 0x7f215819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:09.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 -- 172.21.15.7:0/351851368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f215810e3c0 msgr2=0x7f215819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f215810e3c0 0x7f215819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 -- 172.21.15.7:0/351851368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2158109c90 msgr2=0x7f21581a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2158109c90 0x7f21581a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 -- 172.21.15.7:0/351851368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215819ff80 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2157e00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c00 tx=0x7f214800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148027070 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2148005660 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21581a0210 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.689+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148004930 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.690+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21581ac480 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.691+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215810a7c0 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.693+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f214802a070 con 0x7f21581092e0 2024-04-20T21:32:09.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.695+0000 7f2154c00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 0x7f213c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:09.364 INFO:teuthology.orchestra.run.smithi007.stdout:34359738460 2024-04-20T21:32:09.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.695+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f21480d0f10 con 0x7f21581092e0 2024-04-20T21:32:09.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.695+0000 7f2157400700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 0x7f213c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:09.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.696+0000 7f2157400700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 0x7f213c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2144009750 tx=0x7f2144009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.700+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f214805eff0 con 0x7f21581092e0 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.936+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f21581a0cd0 con 0x7f21581092e0 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.936+0000 7f2154c00700 1 -- 172.21.15.7:0/351851368 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2148013030 con 0x7f21581092e0 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.939+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 msgr2=0x7f213c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.939+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 0x7f213c0a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2144009750 tx=0x7f2144009310 comp rx=0 tx=0).stop 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.939+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 msgr2=0x7f21581a5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.939+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c00 tx=0x7f214800fa70 comp rx=0 tx=0).stop 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 shutdown_connections 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f213c0a3d50 0x7f213c0a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f215810e3c0 0x7f215819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21581092e0 0x7f21581a5620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 --2- 172.21.15.7:0/351851368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2158109c90 0x7f21581a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 >> 172.21.15.7:0/351851368 conn(0x7f2158076220 msgr2=0x7f2158077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 shutdown_connections 2024-04-20T21:32:09.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:07.940+0000 7f215ea00700 1 -- 172.21.15.7:0/351851368 wait complete. 2024-04-20T21:32:10.097 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776163 got 292057776167 for osd.9 2024-04-20T21:32:10.097 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:10.203 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:10 smithi007 ceph-mon[16867]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:32:10.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:10 smithi016 ceph-mon[21409]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:32:10.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:10 smithi186 ceph-mon[22157]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 -- 172.21.15.7:0/2583435106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8107880 msgr2=0x7f87b8107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 --2- 172.21.15.7:0/2583435106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8107880 0x7f87b8107c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009990 tx=0x7f87a8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 -- 172.21.15.7:0/2583435106 shutdown_connections 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 --2- 172.21.15.7:0/2583435106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8102240 0x7f87b810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 --2- 172.21.15.7:0/2583435106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8107880 0x7f87b8107c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 --2- 172.21.15.7:0/2583435106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8101880 0x7f87b8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 -- 172.21.15.7:0/2583435106 >> 172.21.15.7:0/2583435106 conn(0x7f87b80fd370 msgr2=0x7f87b80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 -- 172.21.15.7:0/2583435106 shutdown_connections 2024-04-20T21:32:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.877+0000 7f87b7200700 1 -- 172.21.15.7:0/2583435106 wait complete. 2024-04-20T21:32:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.878+0000 7f87b7200700 1 Processor -- start 2024-04-20T21:32:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.878+0000 7f87b7200700 1 -- start start 2024-04-20T21:32:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 0x7f87b80fd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8102240 0x7f87b80fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 0x7f87b80fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8111c00 con 0x7f87b8101880 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8111a80 con 0x7f87b8107880 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8111d80 con 0x7f87b8102240 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 0x7f87b80fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b5e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 0x7f87b80fd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44270/0 (socket says 172.21.15.7:44270) 2024-04-20T21:32:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b5e00700 1 -- 172.21.15.7:0/751302667 learned_addr learned my addr 172.21.15.7:0/751302667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b5400700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8102240 0x7f87b80fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.879+0000 7f87b6800700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 0x7f87b80fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 -- 172.21.15.7:0/751302667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8102240 msgr2=0x7f87b80fdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8102240 0x7f87b80fdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 -- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 msgr2=0x7f87b80fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 0x7f87b80fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 -- 172.21.15.7:0/751302667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b81a7750 con 0x7f87b8107880 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.880+0000 7f87b6800700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 0x7f87b80fe2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87b000efc0 tx=0x7f87b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.881+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b00092e0 con 0x7f87b8107880 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.881+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87b0004bb0 con 0x7f87b8107880 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.881+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b81a79e0 con 0x7f87b8107880 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.881+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b81a7f20 con 0x7f87b8107880 2024-04-20T21:32:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.881+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b0004280 con 0x7f87b8107880 2024-04-20T21:32:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.882+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b8069740 con 0x7f87b8107880 2024-04-20T21:32:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.884+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f87b001b030 con 0x7f87b8107880 2024-04-20T21:32:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.885+0000 7f87aea00700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 0x7f879c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.886+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f87b00d0300 con 0x7f87b8107880 2024-04-20T21:32:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.886+0000 7f87b5e00700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 0x7f879c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.886+0000 7f87b5e00700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 0x7f879c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:10.888+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f87b0022020 con 0x7f87b8107880 2024-04-20T21:32:11.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.125+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f87b802df90 con 0x7f87b8107880 2024-04-20T21:32:11.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.126+0000 7f87aea00700 1 -- 172.21.15.7:0/751302667 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f87b0010040 con 0x7f87b8107880 2024-04-20T21:32:11.126 INFO:teuthology.orchestra.run.smithi007.stdout:167503724609 2024-04-20T21:32:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 msgr2=0x7f879c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 0x7f879c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87a8006010 tx=0x7f87a800b260 comp rx=0 tx=0).stop 2024-04-20T21:32:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 msgr2=0x7f87b80fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 0x7f87b80fe2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87b000efc0 tx=0x7f87b000c3a0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 shutdown_connections 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b8107880 0x7f87b80fe2a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87b8101880 0x7f87b80fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f879c0a3d50 0x7f879c0a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 --2- 172.21.15.7:0/751302667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87b8102240 0x7f87b80fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 >> 172.21.15.7:0/751302667 conn(0x7f87b80fd370 msgr2=0x7f87b806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 shutdown_connections 2024-04-20T21:32:11.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:11.130+0000 7f87b7200700 1 -- 172.21.15.7:0/751302667 wait complete. 2024-04-20T21:32:11.453 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738460 for osd.0 2024-04-20T21:32:11.453 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:12.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724602 got 167503724609 for osd.5 2024-04-20T21:32:12.155 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:12.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:12 smithi007 ceph-mon[16867]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:12.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:12 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/751302667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:12.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:12 smithi016 ceph-mon[21409]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:12.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:12 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/751302667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:12.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:12 smithi186 ceph-mon[22157]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:12 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/751302667' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-20T21:32:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.031+0000 7f0166e00700 1 -- 172.21.15.7:0/2520750977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016010cf70 msgr2=0x7f016010f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.031+0000 7f0166e00700 1 --2- 172.21.15.7:0/2520750977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016010cf70 0x7f016010f430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f015800b0a0 tx=0x7f015800b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 -- 172.21.15.7:0/2520750977 shutdown_connections 2024-04-20T21:32:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 --2- 172.21.15.7:0/2520750977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016010cf70 0x7f016010f430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 --2- 172.21.15.7:0/2520750977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0160103b40 0x7f0160103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 --2- 172.21.15.7:0/2520750977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160104460 0x7f016010c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 -- 172.21.15.7:0/2520750977 >> 172.21.15.7:0/2520750977 conn(0x7f01600fd370 msgr2=0x7f01600ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 -- 172.21.15.7:0/2520750977 shutdown_connections 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.032+0000 7f0166e00700 1 -- 172.21.15.7:0/2520750977 wait complete. 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.033+0000 7f0166e00700 1 Processor -- start 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.033+0000 7f0166e00700 1 -- start start 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160103b40 0x7f01601017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016010cf70 0x7f0160102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0160112d00 con 0x7f016010cf70 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0160112b80 con 0x7f0160104460 2024-04-20T21:32:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0160112e80 con 0x7f0160103b40 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0166400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016010cf70 0x7f0160102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48060/0 (socket says 172.21.15.7:48060) 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160103b40 0x7f01601017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 -- 172.21.15.7:0/1751165458 learned_addr learned my addr 172.21.15.7:0/1751165458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 -- 172.21.15.7:0/1751165458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160103b40 msgr2=0x7f01601017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160103b40 0x7f01601017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 -- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016010cf70 msgr2=0x7f0160102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016010cf70 0x7f0160102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.034+0000 7f0165000700 1 -- 172.21.15.7:0/1751165458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601a7620 con 0x7f0160104460 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f0165000700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f014c00d750 tx=0x7f014c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c016070 con 0x7f0160104460 2024-04-20T21:32:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601a7910 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f014c00fd70 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c00e3b0 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.035+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601a7ea0 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.036+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0160069740 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.038+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f014c00f890 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.040+0000 7f015e000700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 0x7f01480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.040+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f014c0d00b0 con 0x7f0160104460 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.040+0000 7f0165a00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 0x7f01480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.041+0000 7f0165a00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 0x7f01480a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0154009c60 tx=0x7f0154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.044+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f014c010460 con 0x7f0160104460 2024-04-20T21:32:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.276+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f016002df90 con 0x7f0160104460 2024-04-20T21:32:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.277+0000 7f015e000700 1 -- 172.21.15.7:0/1751165458 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f014c09c900 con 0x7f0160104460 2024-04-20T21:32:13.277 INFO:teuthology.orchestra.run.smithi007.stdout:201863462969 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 msgr2=0x7f01480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 0x7f01480a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0154009c60 tx=0x7f0154009260 comp rx=0 tx=0).stop 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 msgr2=0x7f0160101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f014c00d750 tx=0x7f014c00da60 comp rx=0 tx=0).stop 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 shutdown_connections 2024-04-20T21:32:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01480a3d50 0x7f01480a6210 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0160103b40 0x7f01601017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016010cf70 0x7f0160102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 --2- 172.21.15.7:0/1751165458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0160104460 0x7f0160101ce0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 >> 172.21.15.7:0/1751165458 conn(0x7f01600fd370 msgr2=0x7f01600fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 shutdown_connections 2024-04-20T21:32:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:13.280+0000 7f0166e00700 1 -- 172.21.15.7:0/1751165458 wait complete. 2024-04-20T21:32:14.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462964 got 201863462969 for osd.6 2024-04-20T21:32:14.227 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:14.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:14 smithi007 ceph-mon[16867]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:14.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:14 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1751165458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:14.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:14 smithi016 ceph-mon[21409]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:14.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:14 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1751165458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:14.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:14 smithi186 ceph-mon[22157]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:14.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:14 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1751165458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-20T21:32:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 -- 172.21.15.7:0/4185354227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 msgr2=0x7febf810f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 --2- 172.21.15.7:0/4185354227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf810f420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe8009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 -- 172.21.15.7:0/4185354227 shutdown_connections 2024-04-20T21:32:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 --2- 172.21.15.7:0/4185354227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8109950 0x7febf8115df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 --2- 172.21.15.7:0/4185354227 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf810f420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 --2- 172.21.15.7:0/4185354227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8108f90 0x7febf8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.413+0000 7febf7200700 1 -- 172.21.15.7:0/4185354227 >> 172.21.15.7:0/4185354227 conn(0x7febf80787e0 msgr2=0x7febf8107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.414+0000 7febf7200700 1 -- 172.21.15.7:0/4185354227 shutdown_connections 2024-04-20T21:32:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.414+0000 7febf7200700 1 -- 172.21.15.7:0/4185354227 wait complete. 2024-04-20T21:32:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 Processor -- start 2024-04-20T21:32:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 -- start start 2024-04-20T21:32:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf81a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febf8119340 con 0x7febf810f040 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febf81191c0 con 0x7febf8109950 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.415+0000 7febf7200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7febf81194c0 con 0x7febf8108f90 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf6800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf81a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf6800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf81a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38132/0 (socket says 172.21.15.7:38132) 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48064/0 (socket says 172.21.15.7:48064) 2024-04-20T21:32:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf6800700 1 -- 172.21.15.7:0/2402231970 learned_addr learned my addr 172.21.15.7:0/2402231970 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:59908/0 (socket says 172.21.15.7:59908) 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 -- 172.21.15.7:0/2402231970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 msgr2=0x7febf819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 -- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 msgr2=0x7febf81a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf81a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5e00700 1 -- 172.21.15.7:0/2402231970 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf81a4740 con 0x7febf8108f90 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.416+0000 7febf5400700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febf5e00700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe80057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8027070 con 0x7febf8108f90 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf81a49d0 con 0x7febf8108f90 2024-04-20T21:32:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe800be70 con 0x7febf8108f90 2024-04-20T21:32:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe80173f0 con 0x7febf8108f90 2024-04-20T21:32:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.417+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf81ac2b0 con 0x7febf8108f90 2024-04-20T21:32:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.418+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf8111390 con 0x7febf8108f90 2024-04-20T21:32:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.421+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7febe802a070 con 0x7febf8108f90 2024-04-20T21:32:15.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.422+0000 7febeea00700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 0x7febdc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:15.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.422+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7febe80d15d0 con 0x7febf8108f90 2024-04-20T21:32:15.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.422+0000 7febf5400700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 0x7febdc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:15.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.423+0000 7febf5400700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 0x7febdc0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febf810e7a0 tx=0x7febe4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:15.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.424+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7febe8013070 con 0x7febf8108f90 2024-04-20T21:32:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.657+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7febf81a0a90 con 0x7febf8108f90 2024-04-20T21:32:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.658+0000 7febeea00700 1 -- 172.21.15.7:0/2402231970 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7febe809deb0 con 0x7febf8108f90 2024-04-20T21:32:15.658 INFO:teuthology.orchestra.run.smithi007.stdout:111669149771 2024-04-20T21:32:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 msgr2=0x7febdc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 0x7febdc0a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7febf810e7a0 tx=0x7febe4009260 comp rx=0 tx=0).stop 2024-04-20T21:32:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 msgr2=0x7febf819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:15.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7febe8009990 tx=0x7febe80057c0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 shutdown_connections 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7febdc0a3cd0 0x7febdc0a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7febf8108f90 0x7febf819f730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf810f040 0x7febf81a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 --2- 172.21.15.7:0/2402231970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf8109950 0x7febf819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 >> 172.21.15.7:0/2402231970 conn(0x7febf80787e0 msgr2=0x7febf80799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 shutdown_connections 2024-04-20T21:32:15.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:15.661+0000 7febf7200700 1 -- 172.21.15.7:0/2402231970 wait complete. 2024-04-20T21:32:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:16 smithi007 ceph-mon[16867]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:16.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:16 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2402231970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:16.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:16 smithi016 ceph-mon[21409]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:16.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:16 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2402231970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:16.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:16 smithi186 ceph-mon[22157]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:16.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:16 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2402231970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-20T21:32:17.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149766 got 111669149771 for osd.3 2024-04-20T21:32:17.555 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 -- 172.21.15.7:0/1061594254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 msgr2=0x7f74d4108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 --2- 172.21.15.7:0/1061594254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d4108e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74c40098f0 tx=0x7f74c4009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 -- 172.21.15.7:0/1061594254 shutdown_connections 2024-04-20T21:32:18.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 --2- 172.21.15.7:0/1061594254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406b100 0x7f74d410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 --2- 172.21.15.7:0/1061594254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d4108e20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 --2- 172.21.15.7:0/1061594254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406a760 0x7f74d406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.101+0000 7f74d9200700 1 -- 172.21.15.7:0/1061594254 >> 172.21.15.7:0/1061594254 conn(0x7f74d4078900 msgr2=0x7f74d4078d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.102+0000 7f74d9200700 1 -- 172.21.15.7:0/1061594254 shutdown_connections 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.102+0000 7f74d9200700 1 -- 172.21.15.7:0/1061594254 wait complete. 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 Processor -- start 2024-04-20T21:32:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 -- start start 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 0x7f74d4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112d70 con 0x7f74d4108a40 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112bf0 con 0x7f74d406b100 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.103+0000 7f74d9200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4112ef0 con 0x7f74d406a760 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d1e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d1e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:59916/0 (socket says 172.21.15.7:59916) 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d1e00700 1 -- 172.21.15.7:0/2243752519 learned_addr learned my addr 172.21.15.7:0/2243752519 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74cbe00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 0x7f74d4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 -- 172.21.15.7:0/2243752519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 msgr2=0x7f74d41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 -- 172.21.15.7:0/2243752519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 msgr2=0x7f74d4199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 0x7f74d4199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d2800700 1 -- 172.21.15.7:0/2243752519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d419e2d0 con 0x7f74d4108a40 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74cbe00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 0x7f74d4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.104+0000 7f74d1e00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74d2800700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d419db90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74cc007f10 tx=0x7f74cc0117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc004db0 con 0x7f74d4108a40 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74cc009e20 con 0x7f74d4108a40 2024-04-20T21:32:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d419e5c0 con 0x7f74d4108a40 2024-04-20T21:32:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc01a7d0 con 0x7f74d4108a40 2024-04-20T21:32:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.105+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d4101950 con 0x7f74d4108a40 2024-04-20T21:32:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.106+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d4069740 con 0x7f74d4108a40 2024-04-20T21:32:18.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.109+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 17) ==== 138696+0+0 (secure 0 0 0) 0x7f74cc01a970 con 0x7f74d4108a40 2024-04-20T21:32:18.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.112+0000 7f74caa00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 0x7f74b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:18.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.112+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f74cc0d4560 con 0x7f74d4108a40 2024-04-20T21:32:18.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.112+0000 7f74d1e00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 0x7f74b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:18.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.112+0000 7f74d1e00700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 0x7f74b80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c010 tx=0x7f74c4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.115+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f74cc016070 con 0x7f74d4108a40 2024-04-20T21:32:18.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:18 smithi007 ceph-mon[16867]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.387+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f74d419a620 con 0x7f74d4108a40 2024-04-20T21:32:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.387+0000 7f74caa00700 1 -- 172.21.15.7:0/2243752519 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f74cc019300 con 0x7f74d4108a40 2024-04-20T21:32:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:347892351007 2024-04-20T21:32:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.390+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 msgr2=0x7f74b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.390+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 0x7f74b80a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c010 tx=0x7f74c4019040 comp rx=0 tx=0).stop 2024-04-20T21:32:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.390+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 msgr2=0x7f74d419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.390+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d419db90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74cc007f10 tx=0x7f74cc0117c0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 shutdown_connections 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f74b80a3cd0 0x7f74b80a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f74d406a760 0x7f74d41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74d4108a40 0x7f74d419db90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 --2- 172.21.15.7:0/2243752519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74d406b100 0x7f74d4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 >> 172.21.15.7:0/2243752519 conn(0x7f74d4078900 msgr2=0x7f74d40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.391+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 shutdown_connections 2024-04-20T21:32:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:18.392+0000 7f74d9200700 1 -- 172.21.15.7:0/2243752519 wait complete. 2024-04-20T21:32:18.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:18 smithi016 ceph-mon[21409]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:18 smithi186 ceph-mon[22157]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:19.330 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:19 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2243752519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:19.330 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.330 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:19 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351007 for osd.11 2024-04-20T21:32:19.334 DEBUG:teuthology.parallel:result is None 2024-04-20T21:32:19.334 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-20T21:32:19.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-20T21:32:19.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:19 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2243752519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:19.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:19 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:19 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2243752519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-20T21:32:19.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:19 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:19.822 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:20.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:20 smithi007 ceph-mon[16867]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:32:20.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:20 smithi007 ceph-mon[16867]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-20T21:32:20.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:20.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:20.284 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:20 smithi007 ceph-mon[16867]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:20.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:20 smithi016 ceph-mon[21409]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:32:20.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:20 smithi016 ceph-mon[21409]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-20T21:32:20.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:20.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:20.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:20 smithi016 ceph-mon[21409]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:20.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:20 smithi186 ceph-mon[22157]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:32:20.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:20 smithi186 ceph-mon[22157]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-20T21:32:20.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:32:20.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:32:20.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:20 smithi186 ceph-mon[22157]: from='mgr.14152 172.21.15.7:0/261847680' entity='mgr.x' 2024-04-20T21:32:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 -- 172.21.15.7:0/2148197971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 msgr2=0x7f2cd4111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 --2- 172.21.15.7:0/2148197971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd4111b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b0d0 tx=0x7f2ccc00b3e0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 -- 172.21.15.7:0/2148197971 shutdown_connections 2024-04-20T21:32:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 --2- 172.21.15.7:0/2148197971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4104ed0 0x7f2cd41052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 --2- 172.21.15.7:0/2148197971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd4111b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 --2- 172.21.15.7:0/2148197971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4105880 0x7f2cd41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 -- 172.21.15.7:0/2148197971 >> 172.21.15.7:0/2148197971 conn(0x7f2cd4100760 msgr2=0x7f2cd4102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:21.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 -- 172.21.15.7:0/2148197971 shutdown_connections 2024-04-20T21:32:21.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.954+0000 7f2cdb200700 1 -- 172.21.15.7:0/2148197971 wait complete. 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.955+0000 7f2cdb200700 1 Processor -- start 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.955+0000 7f2cdb200700 1 -- start start 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4104ed0 0x7f2cd419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4105880 0x7f2cd419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:21.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4114fe0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4114e60 con 0x7f2cd4104ed0 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cdb200700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4115160 con 0x7f2cd4105880 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cd9200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cd9200700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38148/0 (socket says 172.21.15.7:38148) 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cd9200700 1 -- 172.21.15.7:0/3087431092 learned_addr learned my addr 172.21.15.7:0/3087431092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cd3e00700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4104ed0 0x7f2cd419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.956+0000 7f2cd3400700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4105880 0x7f2cd419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 -- 172.21.15.7:0/3087431092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4105880 msgr2=0x7f2cd419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4105880 0x7f2cd419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 -- 172.21.15.7:0/3087431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4104ed0 msgr2=0x7f2cd419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4104ed0 0x7f2cd419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:21.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 -- 172.21.15.7:0/3087431092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd41a0460 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd9200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00d010 tx=0x7f2ccc0079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc003c00 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc022070 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc01b6c0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.957+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd41a06f0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.958+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd41a7fd0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.959+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd41063b0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.961+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2ccc009ba0 con 0x7f2cd4109fb0 2024-04-20T21:32:21.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.962+0000 7f2cd0c00700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 0x7f2cb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:21.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.962+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2ccc00f020 con 0x7f2cd4109fb0 2024-04-20T21:32:21.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.962+0000 7f2cd3e00700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 0x7f2cb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.963+0000 7f2cd3e00700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 0x7f2cb80a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009680 tx=0x7f2cc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:21.967+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2ccc09dc70 con 0x7f2cd4109fb0 2024-04-20T21:32:22.193 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:22 smithi007 ceph-mon[16867]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-20T21:32:22.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.193+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cd419c7b0 con 0x7f2cb80a3c40 2024-04-20T21:32:22.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.202+0000 7f2cd0c00700 1 -- 172.21.15.7:0/3087431092 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259046 (secure 0 0 0) 0x7f2cd419c7b0 con 0x7f2cb80a3c40 2024-04-20T21:32:22.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:22.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 msgr2=0x7f2cb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 0x7f2cb80a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009680 tx=0x7f2cc4009260 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 msgr2=0x7f2cd419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00d010 tx=0x7f2ccc0079a0 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 shutdown_connections 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cd4109fb0 0x7f2cd419fd20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd4104ed0 0x7f2cd419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f2cb80a3c40 0x7f2cb80a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 --2- 172.21.15.7:0/3087431092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cd4105880 0x7f2cd419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 >> 172.21.15.7:0/3087431092 conn(0x7f2cd4100760 msgr2=0x7f2cd4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 shutdown_connections 2024-04-20T21:32:22.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:22.210+0000 7f2cdb200700 1 -- 172.21.15.7:0/3087431092 wait complete. 2024-04-20T21:32:22.211 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:32:22.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:22 smithi016 ceph-mon[21409]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-20T21:32:22.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:22 smithi186 ceph-mon[22157]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-20T21:32:23.126 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-20T21:32:20.390224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656257,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":649,"num_read_kb":446,"num_write":348,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352564,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589260,"statfs":{"total":1151940427776,"available":1151579402240,"internally_reserved":0,"allocated":11943936,"data_stored":7471221,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,0,35],"upper_bound":64},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":41,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004842"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.388717+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:32:18.388717+0000","last_peered":"2024-04-20T21:32:18.388717+0000","last_clean":"2024-04-20T21:32:18.388717+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:32:18.388717+0000","last_undegraded":"2024-04-20T21:32:18.388717+0000","last_fullsized":"2024-04-20T21:32:18.388717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.153828+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:31:50.153828+0000","last_peered":"2024-04-20T21:31:50.153828+0000","last_clean":"2024-04-20T21:31:50.153828+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:31:50.153828+0000","last_undegraded":"2024-04-20T21:31:50.153828+0000","last_fullsized":"2024-04-20T21:31:50.153828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.153443+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:31:50.153443+0000","last_peered":"2024-04-20T21:31:50.153443+0000","last_clean":"2024-04-20T21:31:50.153443+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:31:50.153443+0000","last_undegraded":"2024-04-20T21:31:50.153443+0000","last_fullsized":"2024-04-20T21:31:50.153443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.353589+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:32:18.353589+0000","last_peered":"2024-04-20T21:32:18.353589+0000","last_clean":"2024-04-20T21:32:18.353589+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:32:18.353589+0000","last_undegraded":"2024-04-20T21:32:18.353589+0000","last_fullsized":"2024-04-20T21:32:18.353589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142901+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:31:50.142901+0000","last_peered":"2024-04-20T21:31:50.142901+0000","last_clean":"2024-04-20T21:31:50.142901+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:31:50.142901+0000","last_undegraded":"2024-04-20T21:31:50.142901+0000","last_fullsized":"2024-04-20T21:31:50.142901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373944+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:32:18.373944+0000","last_peered":"2024-04-20T21:32:18.373944+0000","last_clean":"2024-04-20T21:32:18.373944+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:32:18.373944+0000","last_undegraded":"2024-04-20T21:32:18.373944+0000","last_fullsized":"2024-04-20T21:32:18.373944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365525+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:32:18.365525+0000","last_peered":"2024-04-20T21:32:18.365525+0000","last_clean":"2024-04-20T21:32:18.365525+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:32:18.365525+0000","last_undegraded":"2024-04-20T21:32:18.365525+0000","last_fullsized":"2024-04-20T21:32:18.365525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143063+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:31:50.143063+0000","last_peered":"2024-04-20T21:31:50.143063+0000","last_clean":"2024-04-20T21:31:50.143063+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:31:50.143063+0000","last_undegraded":"2024-04-20T21:31:50.143063+0000","last_fullsized":"2024-04-20T21:31:50.143063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373846+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:32:18.373846+0000","last_peered":"2024-04-20T21:32:18.373846+0000","last_clean":"2024-04-20T21:32:18.373846+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:32:18.373846+0000","last_undegraded":"2024-04-20T21:32:18.373846+0000","last_fullsized":"2024-04-20T21:32:18.373846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.374449+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:32:18.374449+0000","last_peered":"2024-04-20T21:32:18.374449+0000","last_clean":"2024-04-20T21:32:18.374449+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:32:18.374449+0000","last_undegraded":"2024-04-20T21:32:18.374449+0000","last_fullsized":"2024-04-20T21:32:18.374449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.379664+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:32:18.379664+0000","last_peered":"2024-04-20T21:32:18.379664+0000","last_clean":"2024-04-20T21:32:18.379664+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:32:18.379664+0000","last_undegraded":"2024-04-20T21:32:18.379664+0000","last_fullsized":"2024-04-20T21:32:18.379664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.370940+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:32:18.370940+0000","last_peered":"2024-04-20T21:32:18.370940+0000","last_clean":"2024-04-20T21:32:18.370940+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:32:18.370940+0000","last_undegraded":"2024-04-20T21:32:18.370940+0000","last_fullsized":"2024-04-20T21:32:18.370940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.145954+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:31:50.145954+0000","last_peered":"2024-04-20T21:31:50.145954+0000","last_clean":"2024-04-20T21:31:50.145954+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:31:50.145954+0000","last_undegraded":"2024-04-20T21:31:50.145954+0000","last_fullsized":"2024-04-20T21:31:50.145954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.151176+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:31:50.151176+0000","last_peered":"2024-04-20T21:31:50.151176+0000","last_clean":"2024-04-20T21:31:50.151176+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:31:50.151176+0000","last_undegraded":"2024-04-20T21:31:50.151176+0000","last_fullsized":"2024-04-20T21:31:50.151176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.151026+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:31:50.151026+0000","last_peered":"2024-04-20T21:31:50.151026+0000","last_clean":"2024-04-20T21:31:50.151026+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:31:50.151026+0000","last_undegraded":"2024-04-20T21:31:50.151026+0000","last_fullsized":"2024-04-20T21:31:50.151026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.393933+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:32:18.393933+0000","last_peered":"2024-04-20T21:32:18.393933+0000","last_clean":"2024-04-20T21:32:18.393933+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:32:18.393933+0000","last_undegraded":"2024-04-20T21:32:18.393933+0000","last_fullsized":"2024-04-20T21:32:18.393933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.388944+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:32:18.388944+0000","last_peered":"2024-04-20T21:32:18.388944+0000","last_clean":"2024-04-20T21:32:18.388944+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:32:18.388944+0000","last_undegraded":"2024-04-20T21:32:18.388944+0000","last_fullsized":"2024-04-20T21:32:18.388944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139583+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:31:50.139583+0000","last_peered":"2024-04-20T21:31:50.139583+0000","last_clean":"2024-04-20T21:31:50.139583+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:31:50.139583+0000","last_undegraded":"2024-04-20T21:31:50.139583+0000","last_fullsized":"2024-04-20T21:31:50.139583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146774+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:31:50.146774+0000","last_peered":"2024-04-20T21:31:50.146774+0000","last_clean":"2024-04-20T21:31:50.146774+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:31:50.146774+0000","last_undegraded":"2024-04-20T21:31:50.146774+0000","last_fullsized":"2024-04-20T21:31:50.146774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139580+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:31:50.139580+0000","last_peered":"2024-04-20T21:31:50.139580+0000","last_clean":"2024-04-20T21:31:50.139580+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:31:50.139580+0000","last_undegraded":"2024-04-20T21:31:50.139580+0000","last_fullsized":"2024-04-20T21:31:50.139580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.144143+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:31:50.144143+0000","last_peered":"2024-04-20T21:31:50.144143+0000","last_clean":"2024-04-20T21:31:50.144143+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:31:50.144143+0000","last_undegraded":"2024-04-20T21:31:50.144143+0000","last_fullsized":"2024-04-20T21:31:50.144143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143796+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:31:50.143796+0000","last_peered":"2024-04-20T21:31:50.143796+0000","last_clean":"2024-04-20T21:31:50.143796+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:31:50.143796+0000","last_undegraded":"2024-04-20T21:31:50.143796+0000","last_fullsized":"2024-04-20T21:31:50.143796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.153523+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:31:50.153523+0000","last_peered":"2024-04-20T21:31:50.153523+0000","last_clean":"2024-04-20T21:31:50.153523+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:31:50.153523+0000","last_undegraded":"2024-04-20T21:31:50.153523+0000","last_fullsized":"2024-04-20T21:31:50.153523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143474+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:31:50.143474+0000","last_peered":"2024-04-20T21:31:50.143474+0000","last_clean":"2024-04-20T21:31:50.143474+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:31:50.143474+0000","last_undegraded":"2024-04-20T21:31:50.143474+0000","last_fullsized":"2024-04-20T21:31:50.143474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390550+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:32:18.390550+0000","last_peered":"2024-04-20T21:32:18.390550+0000","last_clean":"2024-04-20T21:32:18.390550+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:32:18.390550+0000","last_undegraded":"2024-04-20T21:32:18.390550+0000","last_fullsized":"2024-04-20T21:32:18.390550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.149839+0000","last_change":"2024-04-20T21:31:50.149839+0000","last_active":"2024-04-20T21:31:50.149839+0000","last_peered":"2024-04-20T21:31:50.149839+0000","last_clean":"2024-04-20T21:31:50.149839+0000","last_became_active":"2024-04-20T21:31:50.145443+0000","last_became_peered":"2024-04-20T21:31:50.145443+0000","last_unstale":"2024-04-20T21:31:50.149839+0000","last_undegraded":"2024-04-20T21:31:50.149839+0000","last_fullsized":"2024-04-20T21:31:50.149839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:56.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143859+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:31:50.143859+0000","last_peered":"2024-04-20T21:31:50.143859+0000","last_clean":"2024-04-20T21:31:50.143859+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:31:50.143859+0000","last_undegraded":"2024-04-20T21:31:50.143859+0000","last_fullsized":"2024-04-20T21:31:50.143859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:16.529611+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:32:16.529611+0000","last_peered":"2024-04-20T21:32:16.529611+0000","last_clean":"2024-04-20T21:32:16.529611+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:32:16.529611+0000","last_undegraded":"2024-04-20T21:32:16.529611+0000","last_fullsized":"2024-04-20T21:32:16.529611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142553+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:31:50.142553+0000","last_peered":"2024-04-20T21:31:50.142553+0000","last_clean":"2024-04-20T21:31:50.142553+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:31:50.142553+0000","last_undegraded":"2024-04-20T21:31:50.142553+0000","last_fullsized":"2024-04-20T21:31:50.142553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.138880+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:31:50.138880+0000","last_peered":"2024-04-20T21:31:50.138880+0000","last_clean":"2024-04-20T21:31:50.138880+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:31:50.138880+0000","last_undegraded":"2024-04-20T21:31:50.138880+0000","last_fullsized":"2024-04-20T21:31:50.138880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142718+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:31:50.142718+0000","last_peered":"2024-04-20T21:31:50.142718+0000","last_clean":"2024-04-20T21:31:50.142718+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:31:50.142718+0000","last_undegraded":"2024-04-20T21:31:50.142718+0000","last_fullsized":"2024-04-20T21:31:50.142718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146573+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:31:50.146573+0000","last_peered":"2024-04-20T21:31:50.146573+0000","last_clean":"2024-04-20T21:31:50.146573+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:31:50.146573+0000","last_undegraded":"2024-04-20T21:31:50.146573+0000","last_fullsized":"2024-04-20T21:31:50.146573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143270+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:31:50.143270+0000","last_peered":"2024-04-20T21:31:50.143270+0000","last_clean":"2024-04-20T21:31:50.143270+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:31:50.143270+0000","last_undegraded":"2024-04-20T21:31:50.143270+0000","last_fullsized":"2024-04-20T21:31:50.143270+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142369+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:31:50.142369+0000","last_peered":"2024-04-20T21:31:50.142369+0000","last_clean":"2024-04-20T21:31:50.142369+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:31:50.142369+0000","last_undegraded":"2024-04-20T21:31:50.142369+0000","last_fullsized":"2024-04-20T21:31:50.142369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139097+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:31:50.139097+0000","last_peered":"2024-04-20T21:31:50.139097+0000","last_clean":"2024-04-20T21:31:50.139097+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:31:50.139097+0000","last_undegraded":"2024-04-20T21:31:50.139097+0000","last_fullsized":"2024-04-20T21:31:50.139097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.322505+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:32:18.322505+0000","last_peered":"2024-04-20T21:32:18.322505+0000","last_clean":"2024-04-20T21:32:18.322505+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:32:18.322505+0000","last_undegraded":"2024-04-20T21:32:18.322505+0000","last_fullsized":"2024-04-20T21:32:18.322505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.377511+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:32:18.377511+0000","last_peered":"2024-04-20T21:32:18.377511+0000","last_clean":"2024-04-20T21:32:18.377511+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:32:18.377511+0000","last_undegraded":"2024-04-20T21:32:18.377511+0000","last_fullsized":"2024-04-20T21:32:18.377511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143688+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:31:50.143688+0000","last_peered":"2024-04-20T21:31:50.143688+0000","last_clean":"2024-04-20T21:31:50.143688+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:31:50.143688+0000","last_undegraded":"2024-04-20T21:31:50.143688+0000","last_fullsized":"2024-04-20T21:31:50.143688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.153442+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:31:50.153442+0000","last_peered":"2024-04-20T21:31:50.153442+0000","last_clean":"2024-04-20T21:31:50.153442+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:31:50.153442+0000","last_undegraded":"2024-04-20T21:31:50.153442+0000","last_fullsized":"2024-04-20T21:31:50.153442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143991+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:31:50.143991+0000","last_peered":"2024-04-20T21:31:50.143991+0000","last_clean":"2024-04-20T21:31:50.143991+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:31:50.143991+0000","last_undegraded":"2024-04-20T21:31:50.143991+0000","last_fullsized":"2024-04-20T21:31:50.143991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.378958+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:32:18.378958+0000","last_peered":"2024-04-20T21:32:18.378958+0000","last_clean":"2024-04-20T21:32:18.378958+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:32:18.378958+0000","last_undegraded":"2024-04-20T21:32:18.378958+0000","last_fullsized":"2024-04-20T21:32:18.378958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.546633+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:32:18.546633+0000","last_peered":"2024-04-20T21:32:18.546633+0000","last_clean":"2024-04-20T21:32:18.546633+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:32:18.546633+0000","last_undegraded":"2024-04-20T21:32:18.546633+0000","last_fullsized":"2024-04-20T21:32:18.546633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61122,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":172,"num_read_kb":146,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143203+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:31:50.143203+0000","last_peered":"2024-04-20T21:31:50.143203+0000","last_clean":"2024-04-20T21:31:50.143203+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:31:50.143203+0000","last_undegraded":"2024-04-20T21:31:50.143203+0000","last_fullsized":"2024-04-20T21:31:50.143203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146247+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:31:50.146247+0000","last_peered":"2024-04-20T21:31:50.146247+0000","last_clean":"2024-04-20T21:31:50.146247+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:31:50.146247+0000","last_undegraded":"2024-04-20T21:31:50.146247+0000","last_fullsized":"2024-04-20T21:31:50.146247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.145246+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:31:50.145246+0000","last_peered":"2024-04-20T21:31:50.145246+0000","last_clean":"2024-04-20T21:31:50.145246+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:31:50.145246+0000","last_undegraded":"2024-04-20T21:31:50.145246+0000","last_fullsized":"2024-04-20T21:31:50.145246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142466+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:31:50.142466+0000","last_peered":"2024-04-20T21:31:50.142466+0000","last_clean":"2024-04-20T21:31:50.142466+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:31:50.142466+0000","last_undegraded":"2024-04-20T21:31:50.142466+0000","last_fullsized":"2024-04-20T21:31:50.142466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.384499+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:32:18.384499+0000","last_peered":"2024-04-20T21:32:18.384499+0000","last_clean":"2024-04-20T21:32:18.384499+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:32:18.384499+0000","last_undegraded":"2024-04-20T21:32:18.384499+0000","last_fullsized":"2024-04-20T21:32:18.384499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:17.549476+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:32:17.549476+0000","last_peered":"2024-04-20T21:32:17.549476+0000","last_clean":"2024-04-20T21:32:17.549476+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:32:17.549476+0000","last_undegraded":"2024-04-20T21:32:17.549476+0000","last_fullsized":"2024-04-20T21:32:17.549476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.372455+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:32:18.372455+0000","last_peered":"2024-04-20T21:32:18.372455+0000","last_clean":"2024-04-20T21:32:18.372455+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:32:18.372455+0000","last_undegraded":"2024-04-20T21:32:18.372455+0000","last_fullsized":"2024-04-20T21:32:18.372455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.475713+0000","last_change":"2024-04-20T21:31:50.475713+0000","last_active":"2024-04-20T21:31:50.475713+0000","last_peered":"2024-04-20T21:31:50.475713+0000","last_clean":"2024-04-20T21:31:50.475713+0000","last_became_active":"2024-04-20T21:31:50.463790+0000","last_became_peered":"2024-04-20T21:31:50.463790+0000","last_unstale":"2024-04-20T21:31:50.475713+0000","last_undegraded":"2024-04-20T21:31:50.475713+0000","last_fullsized":"2024-04-20T21:31:50.475713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:30:43.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.141885+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:31:50.141885+0000","last_peered":"2024-04-20T21:31:50.141885+0000","last_clean":"2024-04-20T21:31:50.141885+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:31:50.141885+0000","last_undegraded":"2024-04-20T21:31:50.141885+0000","last_fullsized":"2024-04-20T21:31:50.141885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143496+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:31:50.143496+0000","last_peered":"2024-04-20T21:31:50.143496+0000","last_clean":"2024-04-20T21:31:50.143496+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:31:50.143496+0000","last_undegraded":"2024-04-20T21:31:50.143496+0000","last_fullsized":"2024-04-20T21:31:50.143496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365054+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:32:18.365054+0000","last_peered":"2024-04-20T21:32:18.365054+0000","last_clean":"2024-04-20T21:32:18.365054+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:32:18.365054+0000","last_undegraded":"2024-04-20T21:32:18.365054+0000","last_fullsized":"2024-04-20T21:32:18.365054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139168+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:31:50.139168+0000","last_peered":"2024-04-20T21:31:50.139168+0000","last_clean":"2024-04-20T21:31:50.139168+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:31:50.139168+0000","last_undegraded":"2024-04-20T21:31:50.139168+0000","last_fullsized":"2024-04-20T21:31:50.139168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383045+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:32:18.383045+0000","last_peered":"2024-04-20T21:32:18.383045+0000","last_clean":"2024-04-20T21:32:18.383045+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:32:18.383045+0000","last_undegraded":"2024-04-20T21:32:18.383045+0000","last_fullsized":"2024-04-20T21:32:18.383045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143410+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:31:50.143410+0000","last_peered":"2024-04-20T21:31:50.143410+0000","last_clean":"2024-04-20T21:31:50.143410+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:31:50.143410+0000","last_undegraded":"2024-04-20T21:31:50.143410+0000","last_fullsized":"2024-04-20T21:31:50.143410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373539+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:32:18.373539+0000","last_peered":"2024-04-20T21:32:18.373539+0000","last_clean":"2024-04-20T21:32:18.373539+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:32:18.373539+0000","last_undegraded":"2024-04-20T21:32:18.373539+0000","last_fullsized":"2024-04-20T21:32:18.373539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142504+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:31:50.142504+0000","last_peered":"2024-04-20T21:31:50.142504+0000","last_clean":"2024-04-20T21:31:50.142504+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:31:50.142504+0000","last_undegraded":"2024-04-20T21:31:50.142504+0000","last_fullsized":"2024-04-20T21:31:50.142504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":103,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:16.546826+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:32:16.546826+0000","last_peered":"2024-04-20T21:32:16.546826+0000","last_clean":"2024-04-20T21:32:16.546826+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:32:16.546826+0000","last_undegraded":"2024-04-20T21:32:16.546826+0000","last_fullsized":"2024-04-20T21:32:16.546826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.144470+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:31:50.144470+0000","last_peered":"2024-04-20T21:31:50.144470+0000","last_clean":"2024-04-20T21:31:50.144470+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:31:50.144470+0000","last_undegraded":"2024-04-20T21:31:50.144470+0000","last_fullsized":"2024-04-20T21:31:50.144470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.462303+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:31:50.462303+0000","last_peered":"2024-04-20T21:31:50.462303+0000","last_clean":"2024-04-20T21:31:50.462303+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:31:50.462303+0000","last_undegraded":"2024-04-20T21:31:50.462303+0000","last_fullsized":"2024-04-20T21:31:50.462303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143266+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:31:50.143266+0000","last_peered":"2024-04-20T21:31:50.143266+0000","last_clean":"2024-04-20T21:31:50.143266+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:31:50.143266+0000","last_undegraded":"2024-04-20T21:31:50.143266+0000","last_fullsized":"2024-04-20T21:31:50.143266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.152048+0000","last_change":"2024-04-20T21:31:50.152048+0000","last_active":"2024-04-20T21:31:50.152048+0000","last_peered":"2024-04-20T21:31:50.152048+0000","last_clean":"2024-04-20T21:31:50.152048+0000","last_became_active":"2024-04-20T21:31:50.151646+0000","last_became_peered":"2024-04-20T21:31:50.151646+0000","last_unstale":"2024-04-20T21:31:50.152048+0000","last_undegraded":"2024-04-20T21:31:50.152048+0000","last_fullsized":"2024-04-20T21:31:50.152048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:22.729679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143334+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:31:50.143334+0000","last_peered":"2024-04-20T21:31:50.143334+0000","last_clean":"2024-04-20T21:31:50.143334+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:31:50.143334+0000","last_undegraded":"2024-04-20T21:31:50.143334+0000","last_fullsized":"2024-04-20T21:31:50.143334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143212+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:31:50.143212+0000","last_peered":"2024-04-20T21:31:50.143212+0000","last_clean":"2024-04-20T21:31:50.143212+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:31:50.143212+0000","last_undegraded":"2024-04-20T21:31:50.143212+0000","last_fullsized":"2024-04-20T21:31:50.143212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146241+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:31:50.146241+0000","last_peered":"2024-04-20T21:31:50.146241+0000","last_clean":"2024-04-20T21:31:50.146241+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:31:50.146241+0000","last_undegraded":"2024-04-20T21:31:50.146241+0000","last_fullsized":"2024-04-20T21:31:50.146241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.154737+0000","last_change":"2024-04-20T21:31:50.154737+0000","last_active":"2024-04-20T21:31:50.154737+0000","last_peered":"2024-04-20T21:31:50.154737+0000","last_clean":"2024-04-20T21:31:50.154737+0000","last_became_active":"2024-04-20T21:31:50.145747+0000","last_became_peered":"2024-04-20T21:31:50.145747+0000","last_unstale":"2024-04-20T21:31:50.154737+0000","last_undegraded":"2024-04-20T21:31:50.154737+0000","last_fullsized":"2024-04-20T21:31:50.154737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:38.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143989+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:31:50.143989+0000","last_peered":"2024-04-20T21:31:50.143989+0000","last_clean":"2024-04-20T21:31:50.143989+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:31:50.143989+0000","last_undegraded":"2024-04-20T21:31:50.143989+0000","last_fullsized":"2024-04-20T21:31:50.143989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.462637+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:31:50.462637+0000","last_peered":"2024-04-20T21:31:50.462637+0000","last_clean":"2024-04-20T21:31:50.462637+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:31:50.462637+0000","last_undegraded":"2024-04-20T21:31:50.462637+0000","last_fullsized":"2024-04-20T21:31:50.462637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.154111+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:31:50.154111+0000","last_peered":"2024-04-20T21:31:50.154111+0000","last_clean":"2024-04-20T21:31:50.154111+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:31:50.154111+0000","last_undegraded":"2024-04-20T21:31:50.154111+0000","last_fullsized":"2024-04-20T21:31:50.154111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.141458+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:31:50.141458+0000","last_peered":"2024-04-20T21:31:50.141458+0000","last_clean":"2024-04-20T21:31:50.141458+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:31:50.141458+0000","last_undegraded":"2024-04-20T21:31:50.141458+0000","last_fullsized":"2024-04-20T21:31:50.141458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142469+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:31:50.142469+0000","last_peered":"2024-04-20T21:31:50.142469+0000","last_clean":"2024-04-20T21:31:50.142469+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:31:50.142469+0000","last_undegraded":"2024-04-20T21:31:50.142469+0000","last_fullsized":"2024-04-20T21:31:50.142469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143209+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:31:50.143209+0000","last_peered":"2024-04-20T21:31:50.143209+0000","last_clean":"2024-04-20T21:31:50.143209+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:31:50.143209+0000","last_undegraded":"2024-04-20T21:31:50.143209+0000","last_fullsized":"2024-04-20T21:31:50.143209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390083+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:32:18.390083+0000","last_peered":"2024-04-20T21:32:18.390083+0000","last_clean":"2024-04-20T21:32:18.390083+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:32:18.390083+0000","last_undegraded":"2024-04-20T21:32:18.390083+0000","last_fullsized":"2024-04-20T21:32:18.390083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146014+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:31:50.146014+0000","last_peered":"2024-04-20T21:31:50.146014+0000","last_clean":"2024-04-20T21:31:50.146014+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:31:50.146014+0000","last_undegraded":"2024-04-20T21:31:50.146014+0000","last_fullsized":"2024-04-20T21:31:50.146014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143160+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:31:50.143160+0000","last_peered":"2024-04-20T21:31:50.143160+0000","last_clean":"2024-04-20T21:31:50.143160+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:31:50.143160+0000","last_undegraded":"2024-04-20T21:31:50.143160+0000","last_fullsized":"2024-04-20T21:31:50.143160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143015+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:31:50.143015+0000","last_peered":"2024-04-20T21:31:50.143015+0000","last_clean":"2024-04-20T21:31:50.143015+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:31:50.143015+0000","last_undegraded":"2024-04-20T21:31:50.143015+0000","last_fullsized":"2024-04-20T21:31:50.143015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383593+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:32:18.383593+0000","last_peered":"2024-04-20T21:32:18.383593+0000","last_clean":"2024-04-20T21:32:18.383593+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:32:18.383593+0000","last_undegraded":"2024-04-20T21:32:18.383593+0000","last_fullsized":"2024-04-20T21:32:18.383593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146590+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:31:50.146590+0000","last_peered":"2024-04-20T21:31:50.146590+0000","last_clean":"2024-04-20T21:31:50.146590+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:31:50.146590+0000","last_undegraded":"2024-04-20T21:31:50.146590+0000","last_fullsized":"2024-04-20T21:31:50.146590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.379703+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:32:18.379703+0000","last_peered":"2024-04-20T21:32:18.379703+0000","last_clean":"2024-04-20T21:32:18.379703+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:32:18.379703+0000","last_undegraded":"2024-04-20T21:32:18.379703+0000","last_fullsized":"2024-04-20T21:32:18.379703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.144003+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:31:50.144003+0000","last_peered":"2024-04-20T21:31:50.144003+0000","last_clean":"2024-04-20T21:31:50.144003+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:31:50.144003+0000","last_undegraded":"2024-04-20T21:31:50.144003+0000","last_fullsized":"2024-04-20T21:31:50.144003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365838+0000","last_change":"2024-04-20T21:31:50.155956+0000","last_active":"2024-04-20T21:32:18.365838+0000","last_peered":"2024-04-20T21:32:18.365838+0000","last_clean":"2024-04-20T21:32:18.365838+0000","last_became_active":"2024-04-20T21:31:50.152234+0000","last_became_peered":"2024-04-20T21:31:50.152234+0000","last_unstale":"2024-04-20T21:32:18.365838+0000","last_undegraded":"2024-04-20T21:32:18.365838+0000","last_fullsized":"2024-04-20T21:32:18.365838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:21.828525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.158229+0000","last_change":"2024-04-20T21:31:50.158229+0000","last_active":"2024-04-20T21:31:50.158229+0000","last_peered":"2024-04-20T21:31:50.158229+0000","last_clean":"2024-04-20T21:31:50.158229+0000","last_became_active":"2024-04-20T21:31:50.154553+0000","last_became_peered":"2024-04-20T21:31:50.154553+0000","last_unstale":"2024-04-20T21:31:50.158229+0000","last_undegraded":"2024-04-20T21:31:50.158229+0000","last_fullsized":"2024-04-20T21:31:50.158229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:13:49.505978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.367597+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:32:18.367597+0000","last_peered":"2024-04-20T21:32:18.367597+0000","last_clean":"2024-04-20T21:32:18.367597+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:32:18.367597+0000","last_undegraded":"2024-04-20T21:32:18.367597+0000","last_fullsized":"2024-04-20T21:32:18.367597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365922+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:32:18.365922+0000","last_peered":"2024-04-20T21:32:18.365922+0000","last_clean":"2024-04-20T21:32:18.365922+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:32:18.365922+0000","last_undegraded":"2024-04-20T21:32:18.365922+0000","last_fullsized":"2024-04-20T21:32:18.365922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.147349+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:31:50.147349+0000","last_peered":"2024-04-20T21:31:50.147349+0000","last_clean":"2024-04-20T21:31:50.147349+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:31:50.147349+0000","last_undegraded":"2024-04-20T21:31:50.147349+0000","last_fullsized":"2024-04-20T21:31:50.147349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.145847+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:31:50.145847+0000","last_peered":"2024-04-20T21:31:50.145847+0000","last_clean":"2024-04-20T21:31:50.145847+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:31:50.145847+0000","last_undegraded":"2024-04-20T21:31:50.145847+0000","last_fullsized":"2024-04-20T21:31:50.145847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.151498+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:31:50.151498+0000","last_peered":"2024-04-20T21:31:50.151498+0000","last_clean":"2024-04-20T21:31:50.151498+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:31:50.151498+0000","last_undegraded":"2024-04-20T21:31:50.151498+0000","last_fullsized":"2024-04-20T21:31:50.151498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146276+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:31:50.146276+0000","last_peered":"2024-04-20T21:31:50.146276+0000","last_clean":"2024-04-20T21:31:50.146276+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:31:50.146276+0000","last_undegraded":"2024-04-20T21:31:50.146276+0000","last_fullsized":"2024-04-20T21:31:50.146276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.364843+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:32:18.364843+0000","last_peered":"2024-04-20T21:32:18.364843+0000","last_clean":"2024-04-20T21:32:18.364843+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:32:18.364843+0000","last_undegraded":"2024-04-20T21:32:18.364843+0000","last_fullsized":"2024-04-20T21:32:18.364843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.147061+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:31:50.147061+0000","last_peered":"2024-04-20T21:31:50.147061+0000","last_clean":"2024-04-20T21:31:50.147061+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:31:50.147061+0000","last_undegraded":"2024-04-20T21:31:50.147061+0000","last_fullsized":"2024-04-20T21:31:50.147061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143535+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:31:50.143535+0000","last_peered":"2024-04-20T21:31:50.143535+0000","last_clean":"2024-04-20T21:31:50.143535+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:31:50.143535+0000","last_undegraded":"2024-04-20T21:31:50.143535+0000","last_fullsized":"2024-04-20T21:31:50.143535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142424+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:31:50.142424+0000","last_peered":"2024-04-20T21:31:50.142424+0000","last_clean":"2024-04-20T21:31:50.142424+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:31:50.142424+0000","last_undegraded":"2024-04-20T21:31:50.142424+0000","last_fullsized":"2024-04-20T21:31:50.142424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143014+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:31:50.143014+0000","last_peered":"2024-04-20T21:31:50.143014+0000","last_clean":"2024-04-20T21:31:50.143014+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:31:50.143014+0000","last_undegraded":"2024-04-20T21:31:50.143014+0000","last_fullsized":"2024-04-20T21:31:50.143014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.381656+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:32:18.381656+0000","last_peered":"2024-04-20T21:32:18.381656+0000","last_clean":"2024-04-20T21:32:18.381656+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:32:18.381656+0000","last_undegraded":"2024-04-20T21:32:18.381656+0000","last_fullsized":"2024-04-20T21:32:18.381656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.145855+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:31:50.145855+0000","last_peered":"2024-04-20T21:31:50.145855+0000","last_clean":"2024-04-20T21:31:50.145855+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:31:50.145855+0000","last_undegraded":"2024-04-20T21:31:50.145855+0000","last_fullsized":"2024-04-20T21:31:50.145855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.363202+0000","last_change":"2024-04-20T21:31:50.144697+0000","last_active":"2024-04-20T21:32:18.363202+0000","last_peered":"2024-04-20T21:32:18.363202+0000","last_clean":"2024-04-20T21:32:18.363202+0000","last_became_active":"2024-04-20T21:31:50.144294+0000","last_became_peered":"2024-04-20T21:31:50.144294+0000","last_unstale":"2024-04-20T21:32:18.363202+0000","last_undegraded":"2024-04-20T21:32:18.363202+0000","last_fullsized":"2024-04-20T21:32:18.363202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:35.702515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281618,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65655,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":513,"num_read_kb":327,"num_write":165,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":459161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,0,35],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1112296,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":459514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":459317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051920,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":458786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":519452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":460057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":521081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149772,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":458980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":1050931,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:32:23.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-20T21:32:23.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:23 smithi016 ceph-mon[21409]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:23.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:23 smithi186 ceph-mon[22157]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:23.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:23 smithi007 ceph-mon[16867]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:23.662 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:24.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:24 smithi016 ceph-mon[21409]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-20T21:32:24.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:24 smithi186 ceph-mon[22157]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-20T21:32:24.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:24 smithi007 ceph-mon[16867]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.038+0000 7f16f9800700 1 -- 172.21.15.7:0/2642741415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 msgr2=0x7f16f410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.038+0000 7f16f9800700 1 --2- 172.21.15.7:0/2642741415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f410dce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099c0 tx=0x7f16e0009cd0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 -- 172.21.15.7:0/2642741415 shutdown_connections 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 --2- 172.21.15.7:0/2642741415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f41092e0 0x7f16f41096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 --2- 172.21.15.7:0/2642741415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f410e3c0 0x7f16f4115f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 --2- 172.21.15.7:0/2642741415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f410dce0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 -- 172.21.15.7:0/2642741415 >> 172.21.15.7:0/2642741415 conn(0x7f16f4076220 msgr2=0x7f16f4107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 -- 172.21.15.7:0/2642741415 shutdown_connections 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.039+0000 7f16f9800700 1 -- 172.21.15.7:0/2642741415 wait complete. 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 Processor -- start 2024-04-20T21:32:26.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 -- start start 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 0x7f16f41a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f410e3c0 0x7f16f419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41193a0 con 0x7f16f4109c90 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4119220 con 0x7f16f41092e0 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.040+0000 7f16f9800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4119520 con 0x7f16f410e3c0 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39704/0 (socket says 172.21.15.7:39704) 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 -- 172.21.15.7:0/2201499713 learned_addr learned my addr 172.21.15.7:0/2201499713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2a00700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 0x7f16f41a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:26.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f3400700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f410e3c0 0x7f16f419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 -- 172.21.15.7:0/2201499713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f410e3c0 msgr2=0x7f16f419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f410e3c0 0x7f16f419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 -- 172.21.15.7:0/2201499713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 msgr2=0x7f16f41a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 0x7f16f41a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2000700 1 -- 172.21.15.7:0/2201499713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f419ff80 con 0x7f16f4109c90 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.041+0000 7f16f2a00700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 0x7f16f41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16f2000700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006010 tx=0x7f16e0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0003680 con 0x7f16f4109c90 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0003e50 con 0x7f16f4109c90 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e000fe10 con 0x7f16f4109c90 2024-04-20T21:32:26.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f41a0210 con 0x7f16f4109c90 2024-04-20T21:32:26.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.042+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41ac500 con 0x7f16f4109c90 2024-04-20T21:32:26.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.043+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f410a7c0 con 0x7f16f4109c90 2024-04-20T21:32:26.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.045+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f16e000f4f0 con 0x7f16f4109c90 2024-04-20T21:32:26.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.048+0000 7f16eb400700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 0x7f16d80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:26.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.048+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f16e00d0ff0 con 0x7f16f4109c90 2024-04-20T21:32:26.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.048+0000 7f16f2a00700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 0x7f16d80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:26.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.049+0000 7f16f2a00700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 0x7f16d80a6320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16e4011490 tx=0x7f16e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:26.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.053+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f16e0013070 con 0x7f16f4109c90 2024-04-20T21:32:26.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.282+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16f41a0b80 con 0x7f16d80a3e60 2024-04-20T21:32:26.281 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:26 smithi007 ceph-mon[16867]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-20T21:32:26.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.295+0000 7f16eb400700 1 -- 172.21.15.7:0/2201499713 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259156 (secure 0 0 0) 0x7f16f41a0b80 con 0x7f16d80a3e60 2024-04-20T21:32:26.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 msgr2=0x7f16d80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 0x7f16d80a6320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16e4011490 tx=0x7f16e4009260 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 msgr2=0x7f16f41a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006010 tx=0x7f16e0003a90 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 shutdown_connections 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f16d80a3e60 0x7f16d80a6320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16f410e3c0 0x7f16f419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16f41092e0 0x7f16f41a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 --2- 172.21.15.7:0/2201499713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f4109c90 0x7f16f41a5ac0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 >> 172.21.15.7:0/2201499713 conn(0x7f16f4076220 msgr2=0x7f16f40767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.302+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 shutdown_connections 2024-04-20T21:32:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:26.303+0000 7f16f9800700 1 -- 172.21.15.7:0/2201499713 wait complete. 2024-04-20T21:32:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:32:26.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:26 smithi016 ceph-mon[21409]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-20T21:32:26.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:26 smithi186 ceph-mon[22157]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-20T21:32:27.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-04-20T21:32:24.391713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656398,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":681,"num_read_kb":463,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352496,"kb_used_data":11664,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589328,"statfs":{"total":1151940427776,"available":1151579471872,"internally_reserved":0,"allocated":11943936,"data_stored":7472142,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,0,35],"upper_bound":64},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":48,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004450"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.388717+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:32:18.388717+0000","last_peered":"2024-04-20T21:32:18.388717+0000","last_clean":"2024-04-20T21:32:18.388717+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:32:18.388717+0000","last_undegraded":"2024-04-20T21:32:18.388717+0000","last_fullsized":"2024-04-20T21:32:18.388717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.377006+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:32:18.377006+0000","last_peered":"2024-04-20T21:32:18.377006+0000","last_clean":"2024-04-20T21:32:18.377006+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:32:18.377006+0000","last_undegraded":"2024-04-20T21:32:18.377006+0000","last_fullsized":"2024-04-20T21:32:18.377006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.386280+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:32:18.386280+0000","last_peered":"2024-04-20T21:32:18.386280+0000","last_clean":"2024-04-20T21:32:18.386280+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:32:18.386280+0000","last_undegraded":"2024-04-20T21:32:18.386280+0000","last_fullsized":"2024-04-20T21:32:18.386280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.353589+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:32:18.353589+0000","last_peered":"2024-04-20T21:32:18.353589+0000","last_clean":"2024-04-20T21:32:18.353589+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:32:18.353589+0000","last_undegraded":"2024-04-20T21:32:18.353589+0000","last_fullsized":"2024-04-20T21:32:18.353589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383357+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:32:18.383357+0000","last_peered":"2024-04-20T21:32:18.383357+0000","last_clean":"2024-04-20T21:32:18.383357+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:32:18.383357+0000","last_undegraded":"2024-04-20T21:32:18.383357+0000","last_fullsized":"2024-04-20T21:32:18.383357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373944+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:32:18.373944+0000","last_peered":"2024-04-20T21:32:18.373944+0000","last_clean":"2024-04-20T21:32:18.373944+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:32:18.373944+0000","last_undegraded":"2024-04-20T21:32:18.373944+0000","last_fullsized":"2024-04-20T21:32:18.373944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365525+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:32:18.365525+0000","last_peered":"2024-04-20T21:32:18.365525+0000","last_clean":"2024-04-20T21:32:18.365525+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:32:18.365525+0000","last_undegraded":"2024-04-20T21:32:18.365525+0000","last_fullsized":"2024-04-20T21:32:18.365525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.371608+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:32:18.371608+0000","last_peered":"2024-04-20T21:32:18.371608+0000","last_clean":"2024-04-20T21:32:18.371608+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:32:18.371608+0000","last_undegraded":"2024-04-20T21:32:18.371608+0000","last_fullsized":"2024-04-20T21:32:18.371608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373846+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:32:18.373846+0000","last_peered":"2024-04-20T21:32:18.373846+0000","last_clean":"2024-04-20T21:32:18.373846+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:32:18.373846+0000","last_undegraded":"2024-04-20T21:32:18.373846+0000","last_fullsized":"2024-04-20T21:32:18.373846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.374449+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:32:18.374449+0000","last_peered":"2024-04-20T21:32:18.374449+0000","last_clean":"2024-04-20T21:32:18.374449+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:32:18.374449+0000","last_undegraded":"2024-04-20T21:32:18.374449+0000","last_fullsized":"2024-04-20T21:32:18.374449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.379664+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:32:18.379664+0000","last_peered":"2024-04-20T21:32:18.379664+0000","last_clean":"2024-04-20T21:32:18.379664+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:32:18.379664+0000","last_undegraded":"2024-04-20T21:32:18.379664+0000","last_fullsized":"2024-04-20T21:32:18.379664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.370940+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:32:18.370940+0000","last_peered":"2024-04-20T21:32:18.370940+0000","last_clean":"2024-04-20T21:32:18.370940+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:32:18.370940+0000","last_undegraded":"2024-04-20T21:32:18.370940+0000","last_fullsized":"2024-04-20T21:32:18.370940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.387576+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:32:18.387576+0000","last_peered":"2024-04-20T21:32:18.387576+0000","last_clean":"2024-04-20T21:32:18.387576+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:32:18.387576+0000","last_undegraded":"2024-04-20T21:32:18.387576+0000","last_fullsized":"2024-04-20T21:32:18.387576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.391913+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:32:18.391913+0000","last_peered":"2024-04-20T21:32:18.391913+0000","last_clean":"2024-04-20T21:32:18.391913+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:32:18.391913+0000","last_undegraded":"2024-04-20T21:32:18.391913+0000","last_fullsized":"2024-04-20T21:32:18.391913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383818+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:32:18.383818+0000","last_peered":"2024-04-20T21:32:18.383818+0000","last_clean":"2024-04-20T21:32:18.383818+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:32:18.383818+0000","last_undegraded":"2024-04-20T21:32:18.383818+0000","last_fullsized":"2024-04-20T21:32:18.383818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.393933+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:32:18.393933+0000","last_peered":"2024-04-20T21:32:18.393933+0000","last_clean":"2024-04-20T21:32:18.393933+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:32:18.393933+0000","last_undegraded":"2024-04-20T21:32:18.393933+0000","last_fullsized":"2024-04-20T21:32:18.393933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.388944+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:32:18.388944+0000","last_peered":"2024-04-20T21:32:18.388944+0000","last_clean":"2024-04-20T21:32:18.388944+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:32:18.388944+0000","last_undegraded":"2024-04-20T21:32:18.388944+0000","last_fullsized":"2024-04-20T21:32:18.388944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.380993+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:32:18.380993+0000","last_peered":"2024-04-20T21:32:18.380993+0000","last_clean":"2024-04-20T21:32:18.380993+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:32:18.380993+0000","last_undegraded":"2024-04-20T21:32:18.380993+0000","last_fullsized":"2024-04-20T21:32:18.380993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.369320+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:32:18.369320+0000","last_peered":"2024-04-20T21:32:18.369320+0000","last_clean":"2024-04-20T21:32:18.369320+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:32:18.369320+0000","last_undegraded":"2024-04-20T21:32:18.369320+0000","last_fullsized":"2024-04-20T21:32:18.369320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139580+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:31:50.139580+0000","last_peered":"2024-04-20T21:31:50.139580+0000","last_clean":"2024-04-20T21:31:50.139580+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:31:50.139580+0000","last_undegraded":"2024-04-20T21:31:50.139580+0000","last_fullsized":"2024-04-20T21:31:50.139580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.368223+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:32:18.368223+0000","last_peered":"2024-04-20T21:32:18.368223+0000","last_clean":"2024-04-20T21:32:18.368223+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:32:18.368223+0000","last_undegraded":"2024-04-20T21:32:18.368223+0000","last_fullsized":"2024-04-20T21:32:18.368223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143796+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:31:50.143796+0000","last_peered":"2024-04-20T21:31:50.143796+0000","last_clean":"2024-04-20T21:31:50.143796+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:31:50.143796+0000","last_undegraded":"2024-04-20T21:31:50.143796+0000","last_fullsized":"2024-04-20T21:31:50.143796+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.387481+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:32:18.387481+0000","last_peered":"2024-04-20T21:32:18.387481+0000","last_clean":"2024-04-20T21:32:18.387481+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:32:18.387481+0000","last_undegraded":"2024-04-20T21:32:18.387481+0000","last_fullsized":"2024-04-20T21:32:18.387481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143474+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:31:50.143474+0000","last_peered":"2024-04-20T21:31:50.143474+0000","last_clean":"2024-04-20T21:31:50.143474+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:31:50.143474+0000","last_undegraded":"2024-04-20T21:31:50.143474+0000","last_fullsized":"2024-04-20T21:31:50.143474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390550+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:32:18.390550+0000","last_peered":"2024-04-20T21:32:18.390550+0000","last_clean":"2024-04-20T21:32:18.390550+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:32:18.390550+0000","last_undegraded":"2024-04-20T21:32:18.390550+0000","last_fullsized":"2024-04-20T21:32:18.390550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.149839+0000","last_change":"2024-04-20T21:31:50.149839+0000","last_active":"2024-04-20T21:31:50.149839+0000","last_peered":"2024-04-20T21:31:50.149839+0000","last_clean":"2024-04-20T21:31:50.149839+0000","last_became_active":"2024-04-20T21:31:50.145443+0000","last_became_peered":"2024-04-20T21:31:50.145443+0000","last_unstale":"2024-04-20T21:31:50.149839+0000","last_undegraded":"2024-04-20T21:31:50.149839+0000","last_fullsized":"2024-04-20T21:31:50.149839+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:56.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.371608+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:32:18.371608+0000","last_peered":"2024-04-20T21:32:18.371608+0000","last_clean":"2024-04-20T21:32:18.371608+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:32:18.371608+0000","last_undegraded":"2024-04-20T21:32:18.371608+0000","last_fullsized":"2024-04-20T21:32:18.371608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:21.537869+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:32:21.537869+0000","last_peered":"2024-04-20T21:32:21.537869+0000","last_clean":"2024-04-20T21:32:21.537869+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:32:21.537869+0000","last_undegraded":"2024-04-20T21:32:21.537869+0000","last_fullsized":"2024-04-20T21:32:21.537869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.358338+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:32:18.358338+0000","last_peered":"2024-04-20T21:32:18.358338+0000","last_clean":"2024-04-20T21:32:18.358338+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:32:18.358338+0000","last_undegraded":"2024-04-20T21:32:18.358338+0000","last_fullsized":"2024-04-20T21:32:18.358338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.138880+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:31:50.138880+0000","last_peered":"2024-04-20T21:31:50.138880+0000","last_clean":"2024-04-20T21:31:50.138880+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:31:50.138880+0000","last_undegraded":"2024-04-20T21:31:50.138880+0000","last_fullsized":"2024-04-20T21:31:50.138880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.376387+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:32:18.376387+0000","last_peered":"2024-04-20T21:32:18.376387+0000","last_clean":"2024-04-20T21:32:18.376387+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:32:18.376387+0000","last_undegraded":"2024-04-20T21:32:18.376387+0000","last_fullsized":"2024-04-20T21:32:18.376387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146573+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:31:50.146573+0000","last_peered":"2024-04-20T21:31:50.146573+0000","last_clean":"2024-04-20T21:31:50.146573+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:31:50.146573+0000","last_undegraded":"2024-04-20T21:31:50.146573+0000","last_fullsized":"2024-04-20T21:31:50.146573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.391711+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:32:18.391711+0000","last_peered":"2024-04-20T21:32:18.391711+0000","last_clean":"2024-04-20T21:32:18.391711+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:32:18.391711+0000","last_undegraded":"2024-04-20T21:32:18.391711+0000","last_fullsized":"2024-04-20T21:32:18.391711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142369+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:31:50.142369+0000","last_peered":"2024-04-20T21:31:50.142369+0000","last_clean":"2024-04-20T21:31:50.142369+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:31:50.142369+0000","last_undegraded":"2024-04-20T21:31:50.142369+0000","last_fullsized":"2024-04-20T21:31:50.142369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373141+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:32:18.373141+0000","last_peered":"2024-04-20T21:32:18.373141+0000","last_clean":"2024-04-20T21:32:18.373141+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:32:18.373141+0000","last_undegraded":"2024-04-20T21:32:18.373141+0000","last_fullsized":"2024-04-20T21:32:18.373141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:22.538754+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:32:22.538754+0000","last_peered":"2024-04-20T21:32:22.538754+0000","last_clean":"2024-04-20T21:32:22.538754+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:32:22.538754+0000","last_undegraded":"2024-04-20T21:32:22.538754+0000","last_fullsized":"2024-04-20T21:32:22.538754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.377511+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:32:18.377511+0000","last_peered":"2024-04-20T21:32:18.377511+0000","last_clean":"2024-04-20T21:32:18.377511+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:32:18.377511+0000","last_undegraded":"2024-04-20T21:32:18.377511+0000","last_fullsized":"2024-04-20T21:32:18.377511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143688+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:31:50.143688+0000","last_peered":"2024-04-20T21:31:50.143688+0000","last_clean":"2024-04-20T21:31:50.143688+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:31:50.143688+0000","last_undegraded":"2024-04-20T21:31:50.143688+0000","last_fullsized":"2024-04-20T21:31:50.143688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.386414+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:32:18.386414+0000","last_peered":"2024-04-20T21:32:18.386414+0000","last_clean":"2024-04-20T21:32:18.386414+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:32:18.386414+0000","last_undegraded":"2024-04-20T21:32:18.386414+0000","last_fullsized":"2024-04-20T21:32:18.386414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143991+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:31:50.143991+0000","last_peered":"2024-04-20T21:31:50.143991+0000","last_clean":"2024-04-20T21:31:50.143991+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:31:50.143991+0000","last_undegraded":"2024-04-20T21:31:50.143991+0000","last_fullsized":"2024-04-20T21:31:50.143991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.378958+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:32:18.378958+0000","last_peered":"2024-04-20T21:32:18.378958+0000","last_clean":"2024-04-20T21:32:18.378958+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:32:18.378958+0000","last_undegraded":"2024-04-20T21:32:18.378958+0000","last_fullsized":"2024-04-20T21:32:18.378958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'33","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:23.554863+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:32:23.554863+0000","last_peered":"2024-04-20T21:32:23.554863+0000","last_clean":"2024-04-20T21:32:23.554863+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:32:23.554863+0000","last_undegraded":"2024-04-20T21:32:23.554863+0000","last_fullsized":"2024-04-20T21:32:23.554863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61122,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":182,"num_read_kb":151,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365932+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:32:18.365932+0000","last_peered":"2024-04-20T21:32:18.365932+0000","last_clean":"2024-04-20T21:32:18.365932+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:32:18.365932+0000","last_undegraded":"2024-04-20T21:32:18.365932+0000","last_fullsized":"2024-04-20T21:32:18.365932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146247+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:31:50.146247+0000","last_peered":"2024-04-20T21:31:50.146247+0000","last_clean":"2024-04-20T21:31:50.146247+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:31:50.146247+0000","last_undegraded":"2024-04-20T21:31:50.146247+0000","last_fullsized":"2024-04-20T21:31:50.146247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.385465+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:32:18.385465+0000","last_peered":"2024-04-20T21:32:18.385465+0000","last_clean":"2024-04-20T21:32:18.385465+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:32:18.385465+0000","last_undegraded":"2024-04-20T21:32:18.385465+0000","last_fullsized":"2024-04-20T21:32:18.385465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.142466+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:31:50.142466+0000","last_peered":"2024-04-20T21:31:50.142466+0000","last_clean":"2024-04-20T21:31:50.142466+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:31:50.142466+0000","last_undegraded":"2024-04-20T21:31:50.142466+0000","last_fullsized":"2024-04-20T21:31:50.142466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.384499+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:32:18.384499+0000","last_peered":"2024-04-20T21:32:18.384499+0000","last_clean":"2024-04-20T21:32:18.384499+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:32:18.384499+0000","last_undegraded":"2024-04-20T21:32:18.384499+0000","last_fullsized":"2024-04-20T21:32:18.384499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:21.559359+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:32:21.559359+0000","last_peered":"2024-04-20T21:32:21.559359+0000","last_clean":"2024-04-20T21:32:21.559359+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:32:21.559359+0000","last_undegraded":"2024-04-20T21:32:21.559359+0000","last_fullsized":"2024-04-20T21:32:21.559359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.372455+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:32:18.372455+0000","last_peered":"2024-04-20T21:32:18.372455+0000","last_clean":"2024-04-20T21:32:18.372455+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:32:18.372455+0000","last_undegraded":"2024-04-20T21:32:18.372455+0000","last_fullsized":"2024-04-20T21:32:18.372455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":59,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.475713+0000","last_change":"2024-04-20T21:31:50.475713+0000","last_active":"2024-04-20T21:31:50.475713+0000","last_peered":"2024-04-20T21:31:50.475713+0000","last_clean":"2024-04-20T21:31:50.475713+0000","last_became_active":"2024-04-20T21:31:50.463790+0000","last_became_peered":"2024-04-20T21:31:50.463790+0000","last_unstale":"2024-04-20T21:31:50.475713+0000","last_undegraded":"2024-04-20T21:31:50.475713+0000","last_fullsized":"2024-04-20T21:31:50.475713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:30:43.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.368573+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:32:18.368573+0000","last_peered":"2024-04-20T21:32:18.368573+0000","last_clean":"2024-04-20T21:32:18.368573+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:32:18.368573+0000","last_undegraded":"2024-04-20T21:32:18.368573+0000","last_fullsized":"2024-04-20T21:32:18.368573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143496+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:31:50.143496+0000","last_peered":"2024-04-20T21:31:50.143496+0000","last_clean":"2024-04-20T21:31:50.143496+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:31:50.143496+0000","last_undegraded":"2024-04-20T21:31:50.143496+0000","last_fullsized":"2024-04-20T21:31:50.143496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365054+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:32:18.365054+0000","last_peered":"2024-04-20T21:32:18.365054+0000","last_clean":"2024-04-20T21:32:18.365054+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:32:18.365054+0000","last_undegraded":"2024-04-20T21:32:18.365054+0000","last_fullsized":"2024-04-20T21:32:18.365054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.139168+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:31:50.139168+0000","last_peered":"2024-04-20T21:31:50.139168+0000","last_clean":"2024-04-20T21:31:50.139168+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:31:50.139168+0000","last_undegraded":"2024-04-20T21:31:50.139168+0000","last_fullsized":"2024-04-20T21:31:50.139168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383045+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:32:18.383045+0000","last_peered":"2024-04-20T21:32:18.383045+0000","last_clean":"2024-04-20T21:32:18.383045+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:32:18.383045+0000","last_undegraded":"2024-04-20T21:32:18.383045+0000","last_fullsized":"2024-04-20T21:32:18.383045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143410+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:31:50.143410+0000","last_peered":"2024-04-20T21:31:50.143410+0000","last_clean":"2024-04-20T21:31:50.143410+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:31:50.143410+0000","last_undegraded":"2024-04-20T21:31:50.143410+0000","last_fullsized":"2024-04-20T21:31:50.143410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373539+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:32:18.373539+0000","last_peered":"2024-04-20T21:32:18.373539+0000","last_clean":"2024-04-20T21:32:18.373539+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:32:18.373539+0000","last_undegraded":"2024-04-20T21:32:18.373539+0000","last_fullsized":"2024-04-20T21:32:18.373539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.362817+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:32:18.362817+0000","last_peered":"2024-04-20T21:32:18.362817+0000","last_clean":"2024-04-20T21:32:18.362817+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:32:18.362817+0000","last_undegraded":"2024-04-20T21:32:18.362817+0000","last_fullsized":"2024-04-20T21:32:18.362817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:21.558578+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:32:21.558578+0000","last_peered":"2024-04-20T21:32:21.558578+0000","last_clean":"2024-04-20T21:32:21.558578+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:32:21.558578+0000","last_undegraded":"2024-04-20T21:32:21.558578+0000","last_fullsized":"2024-04-20T21:32:21.558578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.380105+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:32:18.380105+0000","last_peered":"2024-04-20T21:32:18.380105+0000","last_clean":"2024-04-20T21:32:18.380105+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:32:18.380105+0000","last_undegraded":"2024-04-20T21:32:18.380105+0000","last_fullsized":"2024-04-20T21:32:18.380105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.462303+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:31:50.462303+0000","last_peered":"2024-04-20T21:31:50.462303+0000","last_clean":"2024-04-20T21:31:50.462303+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:31:50.462303+0000","last_undegraded":"2024-04-20T21:31:50.462303+0000","last_fullsized":"2024-04-20T21:31:50.462303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143266+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:31:50.143266+0000","last_peered":"2024-04-20T21:31:50.143266+0000","last_clean":"2024-04-20T21:31:50.143266+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:31:50.143266+0000","last_undegraded":"2024-04-20T21:31:50.143266+0000","last_fullsized":"2024-04-20T21:31:50.143266+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.387019+0000","last_change":"2024-04-20T21:31:50.152048+0000","last_active":"2024-04-20T21:32:18.387019+0000","last_peered":"2024-04-20T21:32:18.387019+0000","last_clean":"2024-04-20T21:32:18.387019+0000","last_became_active":"2024-04-20T21:31:50.151646+0000","last_became_peered":"2024-04-20T21:31:50.151646+0000","last_unstale":"2024-04-20T21:32:18.387019+0000","last_undegraded":"2024-04-20T21:32:18.387019+0000","last_fullsized":"2024-04-20T21:32:18.387019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:22.729679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143334+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:31:50.143334+0000","last_peered":"2024-04-20T21:31:50.143334+0000","last_clean":"2024-04-20T21:31:50.143334+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:31:50.143334+0000","last_undegraded":"2024-04-20T21:31:50.143334+0000","last_fullsized":"2024-04-20T21:31:50.143334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143212+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:31:50.143212+0000","last_peered":"2024-04-20T21:31:50.143212+0000","last_clean":"2024-04-20T21:31:50.143212+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:31:50.143212+0000","last_undegraded":"2024-04-20T21:31:50.143212+0000","last_fullsized":"2024-04-20T21:31:50.143212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390911+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:32:18.390911+0000","last_peered":"2024-04-20T21:32:18.390911+0000","last_clean":"2024-04-20T21:32:18.390911+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:32:18.390911+0000","last_undegraded":"2024-04-20T21:32:18.390911+0000","last_fullsized":"2024-04-20T21:32:18.390911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.155498+0000","last_change":"2024-04-20T21:31:50.154737+0000","last_active":"2024-04-20T21:31:50.155498+0000","last_peered":"2024-04-20T21:31:50.155498+0000","last_clean":"2024-04-20T21:31:50.155498+0000","last_became_active":"2024-04-20T21:31:50.145747+0000","last_became_peered":"2024-04-20T21:31:50.145747+0000","last_unstale":"2024-04-20T21:31:50.155498+0000","last_undegraded":"2024-04-20T21:31:50.155498+0000","last_fullsized":"2024-04-20T21:31:50.155498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:38.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.373878+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:32:18.373878+0000","last_peered":"2024-04-20T21:32:18.373878+0000","last_clean":"2024-04-20T21:32:18.373878+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:32:18.373878+0000","last_undegraded":"2024-04-20T21:32:18.373878+0000","last_fullsized":"2024-04-20T21:32:18.373878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.462637+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:31:50.462637+0000","last_peered":"2024-04-20T21:31:50.462637+0000","last_clean":"2024-04-20T21:31:50.462637+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:31:50.462637+0000","last_undegraded":"2024-04-20T21:31:50.462637+0000","last_fullsized":"2024-04-20T21:31:50.462637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.392573+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:32:18.392573+0000","last_peered":"2024-04-20T21:32:18.392573+0000","last_clean":"2024-04-20T21:32:18.392573+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:32:18.392573+0000","last_undegraded":"2024-04-20T21:32:18.392573+0000","last_fullsized":"2024-04-20T21:32:18.392573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.141458+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:31:50.141458+0000","last_peered":"2024-04-20T21:31:50.141458+0000","last_clean":"2024-04-20T21:31:50.141458+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:31:50.141458+0000","last_undegraded":"2024-04-20T21:31:50.141458+0000","last_fullsized":"2024-04-20T21:31:50.141458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.352746+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:32:18.352746+0000","last_peered":"2024-04-20T21:32:18.352746+0000","last_clean":"2024-04-20T21:32:18.352746+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:32:18.352746+0000","last_undegraded":"2024-04-20T21:32:18.352746+0000","last_fullsized":"2024-04-20T21:32:18.352746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143209+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:31:50.143209+0000","last_peered":"2024-04-20T21:31:50.143209+0000","last_clean":"2024-04-20T21:31:50.143209+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:31:50.143209+0000","last_undegraded":"2024-04-20T21:31:50.143209+0000","last_fullsized":"2024-04-20T21:31:50.143209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390083+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:32:18.390083+0000","last_peered":"2024-04-20T21:32:18.390083+0000","last_clean":"2024-04-20T21:32:18.390083+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:32:18.390083+0000","last_undegraded":"2024-04-20T21:32:18.390083+0000","last_fullsized":"2024-04-20T21:32:18.390083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146014+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:31:50.146014+0000","last_peered":"2024-04-20T21:31:50.146014+0000","last_clean":"2024-04-20T21:31:50.146014+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:31:50.146014+0000","last_undegraded":"2024-04-20T21:31:50.146014+0000","last_fullsized":"2024-04-20T21:31:50.146014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.393568+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:32:18.393568+0000","last_peered":"2024-04-20T21:32:18.393568+0000","last_clean":"2024-04-20T21:32:18.393568+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:32:18.393568+0000","last_undegraded":"2024-04-20T21:32:18.393568+0000","last_fullsized":"2024-04-20T21:32:18.393568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.143015+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:31:50.143015+0000","last_peered":"2024-04-20T21:31:50.143015+0000","last_clean":"2024-04-20T21:31:50.143015+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:31:50.143015+0000","last_undegraded":"2024-04-20T21:31:50.143015+0000","last_fullsized":"2024-04-20T21:31:50.143015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.383593+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:32:18.383593+0000","last_peered":"2024-04-20T21:32:18.383593+0000","last_clean":"2024-04-20T21:32:18.383593+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:32:18.383593+0000","last_undegraded":"2024-04-20T21:32:18.383593+0000","last_fullsized":"2024-04-20T21:32:18.383593+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.146590+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:31:50.146590+0000","last_peered":"2024-04-20T21:31:50.146590+0000","last_clean":"2024-04-20T21:31:50.146590+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:31:50.146590+0000","last_undegraded":"2024-04-20T21:31:50.146590+0000","last_fullsized":"2024-04-20T21:31:50.146590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.379703+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:32:18.379703+0000","last_peered":"2024-04-20T21:32:18.379703+0000","last_clean":"2024-04-20T21:32:18.379703+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:32:18.379703+0000","last_undegraded":"2024-04-20T21:32:18.379703+0000","last_fullsized":"2024-04-20T21:32:18.379703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.144003+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:31:50.144003+0000","last_peered":"2024-04-20T21:31:50.144003+0000","last_clean":"2024-04-20T21:31:50.144003+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:31:50.144003+0000","last_undegraded":"2024-04-20T21:31:50.144003+0000","last_fullsized":"2024-04-20T21:31:50.144003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365838+0000","last_change":"2024-04-20T21:31:50.155956+0000","last_active":"2024-04-20T21:32:18.365838+0000","last_peered":"2024-04-20T21:32:18.365838+0000","last_clean":"2024-04-20T21:32:18.365838+0000","last_became_active":"2024-04-20T21:31:50.152234+0000","last_became_peered":"2024-04-20T21:31:50.152234+0000","last_unstale":"2024-04-20T21:32:18.365838+0000","last_undegraded":"2024-04-20T21:32:18.365838+0000","last_fullsized":"2024-04-20T21:32:18.365838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:21.828525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:31:50.158229+0000","last_change":"2024-04-20T21:31:50.158229+0000","last_active":"2024-04-20T21:31:50.158229+0000","last_peered":"2024-04-20T21:31:50.158229+0000","last_clean":"2024-04-20T21:31:50.158229+0000","last_became_active":"2024-04-20T21:31:50.154553+0000","last_became_peered":"2024-04-20T21:31:50.154553+0000","last_unstale":"2024-04-20T21:31:50.158229+0000","last_undegraded":"2024-04-20T21:31:50.158229+0000","last_fullsized":"2024-04-20T21:31:50.158229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:13:49.505978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.367597+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:32:18.367597+0000","last_peered":"2024-04-20T21:32:18.367597+0000","last_clean":"2024-04-20T21:32:18.367597+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:32:18.367597+0000","last_undegraded":"2024-04-20T21:32:18.367597+0000","last_fullsized":"2024-04-20T21:32:18.367597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.365922+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:32:18.365922+0000","last_peered":"2024-04-20T21:32:18.365922+0000","last_clean":"2024-04-20T21:32:18.365922+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:32:18.365922+0000","last_undegraded":"2024-04-20T21:32:18.365922+0000","last_fullsized":"2024-04-20T21:32:18.365922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.363856+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:32:18.363856+0000","last_peered":"2024-04-20T21:32:18.363856+0000","last_clean":"2024-04-20T21:32:18.363856+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:32:18.363856+0000","last_undegraded":"2024-04-20T21:32:18.363856+0000","last_fullsized":"2024-04-20T21:32:18.363856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.376229+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:32:18.376229+0000","last_peered":"2024-04-20T21:32:18.376229+0000","last_clean":"2024-04-20T21:32:18.376229+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:32:18.376229+0000","last_undegraded":"2024-04-20T21:32:18.376229+0000","last_fullsized":"2024-04-20T21:32:18.376229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.392200+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:32:18.392200+0000","last_peered":"2024-04-20T21:32:18.392200+0000","last_clean":"2024-04-20T21:32:18.392200+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:32:18.392200+0000","last_undegraded":"2024-04-20T21:32:18.392200+0000","last_fullsized":"2024-04-20T21:32:18.392200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.390477+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:32:18.390477+0000","last_peered":"2024-04-20T21:32:18.390477+0000","last_clean":"2024-04-20T21:32:18.390477+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:32:18.390477+0000","last_undegraded":"2024-04-20T21:32:18.390477+0000","last_fullsized":"2024-04-20T21:32:18.390477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.364843+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:32:18.364843+0000","last_peered":"2024-04-20T21:32:18.364843+0000","last_clean":"2024-04-20T21:32:18.364843+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:32:18.364843+0000","last_undegraded":"2024-04-20T21:32:18.364843+0000","last_fullsized":"2024-04-20T21:32:18.364843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.352251+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:32:18.352251+0000","last_peered":"2024-04-20T21:32:18.352251+0000","last_clean":"2024-04-20T21:32:18.352251+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:32:18.352251+0000","last_undegraded":"2024-04-20T21:32:18.352251+0000","last_fullsized":"2024-04-20T21:32:18.352251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.370465+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:32:18.370465+0000","last_peered":"2024-04-20T21:32:18.370465+0000","last_clean":"2024-04-20T21:32:18.370465+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:32:18.370465+0000","last_undegraded":"2024-04-20T21:32:18.370465+0000","last_fullsized":"2024-04-20T21:32:18.370465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.376051+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:32:18.376051+0000","last_peered":"2024-04-20T21:32:18.376051+0000","last_clean":"2024-04-20T21:32:18.376051+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:32:18.376051+0000","last_undegraded":"2024-04-20T21:32:18.376051+0000","last_fullsized":"2024-04-20T21:32:18.376051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.368313+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:32:18.368313+0000","last_peered":"2024-04-20T21:32:18.368313+0000","last_clean":"2024-04-20T21:32:18.368313+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:32:18.368313+0000","last_undegraded":"2024-04-20T21:32:18.368313+0000","last_fullsized":"2024-04-20T21:32:18.368313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.381656+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:32:18.381656+0000","last_peered":"2024-04-20T21:32:18.381656+0000","last_clean":"2024-04-20T21:32:18.381656+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:32:18.381656+0000","last_undegraded":"2024-04-20T21:32:18.381656+0000","last_fullsized":"2024-04-20T21:32:18.381656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.393345+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:32:18.393345+0000","last_peered":"2024-04-20T21:32:18.393345+0000","last_clean":"2024-04-20T21:32:18.393345+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:32:18.393345+0000","last_undegraded":"2024-04-20T21:32:18.393345+0000","last_fullsized":"2024-04-20T21:32:18.393345+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-20T21:32:18.363202+0000","last_change":"2024-04-20T21:31:50.144697+0000","last_active":"2024-04-20T21:32:18.363202+0000","last_peered":"2024-04-20T21:32:18.363202+0000","last_clean":"2024-04-20T21:32:18.363202+0000","last_became_active":"2024-04-20T21:31:50.144294+0000","last_became_peered":"2024-04-20T21:31:50.144294+0000","last_unstale":"2024-04-20T21:32:18.363202+0000","last_undegraded":"2024-04-20T21:32:18.363202+0000","last_fullsized":"2024-04-20T21:32:18.363202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:35.702515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281618,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65796,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":545,"num_read_kb":344,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":459161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,0,35],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1112296,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":459514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":1051920,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":519701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":460057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":521471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":458980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1051072,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:32:27.068 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-20T21:32:27.069 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T21:32:27.069 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-20T21:32:27.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-20T21:32:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:28.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:28 smithi016 ceph-mon[21409]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:28.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:28 smithi016 ceph-mon[21409]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-20T21:32:28.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:28 smithi186 ceph-mon[22157]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:28.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:28 smithi186 ceph-mon[22157]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-20T21:32:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:28 smithi007 ceph-mon[16867]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:32:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:28 smithi007 ceph-mon[16867]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2043602261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4103330 msgr2=0x7fcfa410f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2043602261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4103330 0x7fcfa410f840 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b0a0 tx=0x7fcf9c00b3b0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2043602261 shutdown_connections 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2043602261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4103330 0x7fcfa410f840 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2043602261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4108970 0x7fcfa4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2043602261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4102970 0x7fcfa4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.944+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2043602261 >> 172.21.15.7:0/2043602261 conn(0x7fcfa40fe460 msgr2=0x7fcfa4100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.945+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2043602261 shutdown_connections 2024-04-20T21:32:29.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.945+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2043602261 wait complete. 2024-04-20T21:32:29.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.946+0000 7fcfa3e00700 1 Processor -- start 2024-04-20T21:32:29.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.946+0000 7fcfa3e00700 1 -- start start 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4103330 0x7fcfa419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4108970 0x7fcfa419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4112d20 con 0x7fcfa4108970 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4112ba0 con 0x7fcfa4102970 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3e00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4112ea0 con 0x7fcfa4103330 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa3400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4108970 0x7fcfa419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34872/0 (socket says 172.21.15.7:34872) 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 -- 172.21.15.7:0/2755068795 learned_addr learned my addr 172.21.15.7:0/2755068795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:29.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2000700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4103330 0x7fcfa419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:29.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 -- 172.21.15.7:0/2755068795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4103330 msgr2=0x7fcfa419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4103330 0x7fcfa419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.947+0000 7fcfa2a00700 1 -- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4108970 msgr2=0x7fcfa419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa2a00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4108970 0x7fcfa419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa2a00700 1 -- 172.21.15.7:0/2755068795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41a02a0 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa2a00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400eb30 tx=0x7fcf9400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400cbd0 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf94004500 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94010430 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa41a0590 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.948+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41a7e10 con 0x7fcfa4102970 2024-04-20T21:32:29.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.949+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa4069740 con 0x7fcfa4102970 2024-04-20T21:32:29.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.952+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fcf94003f00 con 0x7fcfa4102970 2024-04-20T21:32:29.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.953+0000 7fcfa8a00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 0x7fcf880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:29.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.953+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcf940d0680 con 0x7fcfa4102970 2024-04-20T21:32:29.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.953+0000 7fcfa2000700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 0x7fcf880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:29.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.954+0000 7fcfa2000700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 0x7fcf880a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4108260 tx=0x7fcf90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:29.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:29.958+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcf9409d030 con 0x7fcfa4102970 2024-04-20T21:32:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.241+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcfa419c5f0 con 0x7fcfa4102970 2024-04-20T21:32:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.242+0000 7fcfa8a00700 1 -- 172.21.15.7:0/2755068795 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fcf9409c930 con 0x7fcfa4102970 2024-04-20T21:32:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:30.241 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 msgr2=0x7fcf880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 0x7fcf880a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4108260 tx=0x7fcf90009260 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 msgr2=0x7fcfa419b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400eb30 tx=0x7fcf9400ee40 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 shutdown_connections 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfa4108970 0x7fcfa419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa4102970 0x7fcfa419b290 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fcf880a3cc0 0x7fcf880a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 --2- 172.21.15.7:0/2755068795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcfa4103330 0x7fcfa419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 >> 172.21.15.7:0/2755068795 conn(0x7fcfa40fe460 msgr2=0x7fcfa410e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 shutdown_connections 2024-04-20T21:32:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:30.245+0000 7fcfa3e00700 1 -- 172.21.15.7:0/2755068795 wait complete. 2024-04-20T21:32:30.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:30 smithi016 ceph-mon[21409]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:32:30.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:30 smithi186 ceph-mon[22157]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:32:30.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:30 smithi007 ceph-mon[16867]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-20T21:32:31.202 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-20T21:32:31.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:32:31.413 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:31 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2755068795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 -- 172.21.15.7:0/1796378976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa681096a0 msgr2=0x7faa68109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 --2- 172.21.15.7:0/1796378976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa681096a0 0x7faa68109b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa50009900 tx=0x7faa5002bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 -- 172.21.15.7:0/1796378976 shutdown_connections 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 --2- 172.21.15.7:0/1796378976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa6810a060 0x7faa681108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 --2- 172.21.15.7:0/1796378976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa681096a0 0x7faa68109b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 --2- 172.21.15.7:0/1796378976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa68108500 0x7faa68108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 -- 172.21.15.7:0/1796378976 >> 172.21.15.7:0/1796378976 conn(0x7faa68077450 msgr2=0x7faa68079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 -- 172.21.15.7:0/1796378976 shutdown_connections 2024-04-20T21:32:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.435+0000 7faa6dc00640 1 -- 172.21.15.7:0/1796378976 wait complete. 2024-04-20T21:32:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 Processor -- start 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 -- start start 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa681096a0 0x7faa6819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa6810a060 0x7faa681a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa68114190 con 0x7faa68108500 2024-04-20T21:32:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa68114010 con 0x7faa681096a0 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa6dc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa68114310 con 0x7faa6810a060 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.437+0000 7faa67e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39730/0 (socket says 172.21.15.7:39730) 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 -- 172.21.15.7:0/883113845 learned_addr learned my addr 172.21.15.7:0/883113845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67400640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa681096a0 0x7faa6819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa6d200640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa6810a060 0x7faa681a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 -- 172.21.15.7:0/883113845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa6810a060 msgr2=0x7faa681a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa6810a060 0x7faa681a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 -- 172.21.15.7:0/883113845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa681096a0 msgr2=0x7faa6819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa681096a0 0x7faa6819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 -- 172.21.15.7:0/883113845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa681a6430 con 0x7faa68108500 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.438+0000 7faa67e00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e9c0 tx=0x7faa5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.439+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5800cd80 con 0x7faa68108500 2024-04-20T21:32:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.439+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa681a6720 con 0x7faa68108500 2024-04-20T21:32:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.439+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa580035f0 con 0x7faa68108500 2024-04-20T21:32:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.439+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa58010690 con 0x7faa68108500 2024-04-20T21:32:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.439+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa681a6c40 con 0x7faa68108500 2024-04-20T21:32:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.440+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6810f700 con 0x7faa68108500 2024-04-20T21:32:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.442+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7faa580037a0 con 0x7faa68108500 2024-04-20T21:32:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.443+0000 7faa64c00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 0x7faa4c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.444+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7faa580d0790 con 0x7faa68108500 2024-04-20T21:32:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.444+0000 7faa67400640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 0x7faa4c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.445+0000 7faa67400640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 0x7faa4c0a6060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa5002c240 tx=0x7faa5002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:31 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2755068795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:32:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.448+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faa5809d140 con 0x7faa68108500 2024-04-20T21:32:31.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:31 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2755068795' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-20T21:32:31.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.655+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa68062b60 con 0x7faa68108500 2024-04-20T21:32:31.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.658+0000 7faa64c00640 1 -- 172.21.15.7:0/883113845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7faa5809ca40 con 0x7faa68108500 2024-04-20T21:32:31.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:31.658 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 msgr2=0x7faa4c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 0x7faa4c0a6060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa5002c240 tx=0x7faa5002df90 comp rx=0 tx=0).stop 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 msgr2=0x7faa6819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faa5800e9c0 tx=0x7faa5800ee80 comp rx=0 tx=0).stop 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 shutdown_connections 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7faa4c0a3ba0 0x7faa4c0a6060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa681096a0 0x7faa6819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa6810a060 0x7faa681a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 --2- 172.21.15.7:0/883113845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa68108500 0x7faa6819e810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 >> 172.21.15.7:0/883113845 conn(0x7faa68077450 msgr2=0x7faa6810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 shutdown_connections 2024-04-20T21:32:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.661+0000 7faa6dc00640 1 -- 172.21.15.7:0/883113845 wait complete. 2024-04-20T21:32:31.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:32:31.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 -- 172.21.15.7:0/3312597735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100f70 msgr2=0x7fc3101013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3312597735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100f70 0x7fc3101013d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009900 tx=0x7fc2f802bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 -- 172.21.15.7:0/3312597735 shutdown_connections 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3312597735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100f70 0x7fc3101013d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3312597735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310101910 0x7fc310101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3312597735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310100610 0x7fc310100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 -- 172.21.15.7:0/3312597735 >> 172.21.15.7:0/3312597735 conn(0x7fc3100fc230 msgr2=0x7fc3100fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:31.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 -- 172.21.15.7:0/3312597735 shutdown_connections 2024-04-20T21:32:31.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.896+0000 7fc30fe00640 1 -- 172.21.15.7:0/3312597735 wait complete. 2024-04-20T21:32:31.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.897+0000 7fc30fe00640 1 Processor -- start 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.897+0000 7fc30fe00640 1 -- start start 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100610 0x7fc31019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310100f70 0x7fc31019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc31010fc90 con 0x7fc310101910 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc31010fb10 con 0x7fc310100610 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc31010fe10 con 0x7fc310100f70 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39742/0 (socket says 172.21.15.7:39742) 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 -- 172.21.15.7:0/3270902068 learned_addr learned my addr 172.21.15.7:0/3270902068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30ea00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100610 0x7fc31019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc306000640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310100f70 0x7fc31019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 -- 172.21.15.7:0/3270902068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310100f70 msgr2=0x7fc31019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310100f70 0x7fc31019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 -- 172.21.15.7:0/3270902068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100610 msgr2=0x7fc31019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100610 0x7fc31019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.898+0000 7fc30f400640 1 -- 172.21.15.7:0/3270902068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3101a1ea0 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc30f400640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc30800eb10 tx=0x7fc30800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004b20 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc308004cc0 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3101a2190 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308010760 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.899+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3101a26b0 con 0x7fc310101910 2024-04-20T21:32:31.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.900+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31010b360 con 0x7fc310101910 2024-04-20T21:32:31.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.902+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc308005020 con 0x7fc310101910 2024-04-20T21:32:31.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.904+0000 7fc314a00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 0x7fc2f40a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.904+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc3080d0ac0 con 0x7fc310101910 2024-04-20T21:32:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.904+0000 7fc30ea00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 0x7fc2f40a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:31.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.905+0000 7fc30ea00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 0x7fc2f40a6060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc300009bd0 tx=0x7fc300005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:31.908+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc30809d470 con 0x7fc310101910 2024-04-20T21:32:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.114+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc310062b60 con 0x7fc310101910 2024-04-20T21:32:32.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.115+0000 7fc314a00640 1 -- 172.21.15.7:0/3270902068 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fc30809cd70 con 0x7fc310101910 2024-04-20T21:32:32.114 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 msgr2=0x7fc2f40a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 0x7fc2f40a6060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc300009bd0 tx=0x7fc300005f90 comp rx=0 tx=0).stop 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 msgr2=0x7fc3101a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc30800eb10 tx=0x7fc30800c2d0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 shutdown_connections 2024-04-20T21:32:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc2f40a3ba0 0x7fc2f40a6060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc310101910 0x7fc3101a1830 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc310100f70 0x7fc31019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3270902068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310100610 0x7fc31019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 >> 172.21.15.7:0/3270902068 conn(0x7fc3100fc230 msgr2=0x7fc3100fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.118+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 shutdown_connections 2024-04-20T21:32:32.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.119+0000 7fc30fe00640 1 -- 172.21.15.7:0/3270902068 wait complete. 2024-04-20T21:32:32.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 -- 172.21.15.7:0/192253639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10074960 msgr2=0x7f9f10074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 --2- 172.21.15.7:0/192253639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10074960 0x7f9f10074dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009900 tx=0x7f9f0002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 -- 172.21.15.7:0/192253639 shutdown_connections 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 --2- 172.21.15.7:0/192253639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10109e50 0x7f9f1010c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 --2- 172.21.15.7:0/192253639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10074960 0x7f9f10074dc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 --2- 172.21.15.7:0/192253639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10075f40 0x7f9f10074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 -- 172.21.15.7:0/192253639 >> 172.21.15.7:0/192253639 conn(0x7f9f100fd530 msgr2=0x7f9f100ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 -- 172.21.15.7:0/192253639 shutdown_connections 2024-04-20T21:32:32.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.362+0000 7f9f17a00640 1 -- 172.21.15.7:0/192253639 wait complete. 2024-04-20T21:32:32.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.363+0000 7f9f17a00640 1 Processor -- start 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 -- start start 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10075f40 0x7f9f1019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10109e50 0x7f9f101a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010fea0 con 0x7f9f10074960 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010fd20 con 0x7f9f10075f40 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f17a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10110020 con 0x7f9f10109e50 2024-04-20T21:32:32.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f15000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f15000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39752/0 (socket says 172.21.15.7:39752) 2024-04-20T21:32:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.364+0000 7f9f15000640 1 -- 172.21.15.7:0/84427689 learned_addr learned my addr 172.21.15.7:0/84427689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10109e50 0x7f9f101a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f0fe00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10075f40 0x7f9f1019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 -- 172.21.15.7:0/84427689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10109e50 msgr2=0x7f9f101a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10109e50 0x7f9f101a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 -- 172.21.15.7:0/84427689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10075f40 msgr2=0x7f9f1019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10075f40 0x7f9f1019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 -- 172.21.15.7:0/84427689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101a2110 con 0x7f9f10074960 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f15000640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9f0 tx=0x7f9f0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.365+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400cd80 con 0x7f9f10074960 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.366+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f04010910 con 0x7f9f10074960 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.366+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101a2400 con 0x7f9f10074960 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.366+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04018990 con 0x7f9f10074960 2024-04-20T21:32:32.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.366+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a2920 con 0x7f9f10074960 2024-04-20T21:32:32.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.367+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f10108c10 con 0x7f9f10074960 2024-04-20T21:32:32.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.369+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f9f04010ac0 con 0x7f9f10074960 2024-04-20T21:32:32.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.370+0000 7f9f0d600640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 0x7f9ef80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.371+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9f040d0750 con 0x7f9f10074960 2024-04-20T21:32:32.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.371+0000 7f9f0fe00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 0x7f9ef80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.371+0000 7f9f0fe00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 0x7f9ef80a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002c2e0 tx=0x7f9f0002c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.374+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9f0409cab0 con 0x7f9f10074960 2024-04-20T21:32:32.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:32 smithi016 ceph-mon[21409]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-20T21:32:32.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:32 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/883113845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:32 smithi186 ceph-mon[22157]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-20T21:32:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:32 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/883113845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:32 smithi007 ceph-mon[16867]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-20T21:32:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:32 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/883113845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.585+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f10062b60 con 0x7f9f10074960 2024-04-20T21:32:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.586+0000 7f9f0d600640 1 -- 172.21.15.7:0/84427689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f9f040a0db0 con 0x7f9f10074960 2024-04-20T21:32:32.585 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 msgr2=0x7f9ef80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 0x7f9ef80a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002c2e0 tx=0x7f9f0002c920 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 msgr2=0x7f9f1019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9f0 tx=0x7f9f0400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 shutdown_connections 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f9ef80a3bb0 0x7f9ef80a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f10075f40 0x7f9f1019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9f10109e50 0x7f9f101a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 --2- 172.21.15.7:0/84427689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f10074960 0x7f9f1019a4f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 >> 172.21.15.7:0/84427689 conn(0x7f9f100fd530 msgr2=0x7f9f10105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 shutdown_connections 2024-04-20T21:32:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.589+0000 7f9f17a00640 1 -- 172.21.15.7:0/84427689 wait complete. 2024-04-20T21:32:32.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:32:32.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.829+0000 7f0207e00640 1 -- 172.21.15.7:0/1563581602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208105170 msgr2=0x7f0208107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.829+0000 7f0207e00640 1 --2- 172.21.15.7:0/1563581602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208105170 0x7f0208107560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009900 tx=0x7f01f402bc50 comp rx=0 tx=0).stop 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 -- 172.21.15.7:0/1563581602 shutdown_connections 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 --2- 172.21.15.7:0/1563581602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208107aa0 0x7f0208109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 --2- 172.21.15.7:0/1563581602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208105170 0x7f0208107560 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 --2- 172.21.15.7:0/1563581602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208102840 0x7f0208104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 -- 172.21.15.7:0/1563581602 >> 172.21.15.7:0/1563581602 conn(0x7f02080fc250 msgr2=0x7f02080fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 -- 172.21.15.7:0/1563581602 shutdown_connections 2024-04-20T21:32:32.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.830+0000 7f0207e00640 1 -- 172.21.15.7:0/1563581602 wait complete. 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 Processor -- start 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 -- start start 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208105170 0x7f02081007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 0x7f0208100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f020810da80 con 0x7f0208107aa0 2024-04-20T21:32:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f020810d900 con 0x7f0208105170 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.831+0000 7f0207e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f020810dc00 con 0x7f0208102840 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0207400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 0x7f0208100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208105170 0x7f02081007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0207400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 0x7f0208100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53728/0 (socket says 172.21.15.7:53728) 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43080/0 (socket says 172.21.15.7:43080) 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0207400640 1 -- 172.21.15.7:0/511877455 learned_addr learned my addr 172.21.15.7:0/511877455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 -- 172.21.15.7:0/511877455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208105170 msgr2=0x7f02081007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208105170 0x7f02081007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 -- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 msgr2=0x7f0208100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 0x7f0208100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 -- 172.21.15.7:0/511877455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02081015c0 con 0x7f0208102840 2024-04-20T21:32:32.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.832+0000 7f0206a00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9f0 tx=0x7f01f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.833+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f800cd80 con 0x7f0208102840 2024-04-20T21:32:32.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.833+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f80035f0 con 0x7f0208102840 2024-04-20T21:32:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.833+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02081a2220 con 0x7f0208102840 2024-04-20T21:32:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.833+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8010690 con 0x7f0208102840 2024-04-20T21:32:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.833+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02081a2740 con 0x7f0208102840 2024-04-20T21:32:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.834+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020804fcf0 con 0x7f0208102840 2024-04-20T21:32:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.836+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f01f8003b30 con 0x7f0208102840 2024-04-20T21:32:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.838+0000 7f01ff400640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 0x7f01ec0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.838+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f01f80d0730 con 0x7f0208102840 2024-04-20T21:32:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.838+0000 7f0206000640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 0x7f01ec0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.839+0000 7f0206000640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 0x7f01ec0a6060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01f402c190 tx=0x7f01f4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:32.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:32.842+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f01f809d0e0 con 0x7f0208102840 2024-04-20T21:32:33.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.051+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0208062b60 con 0x7f0208102840 2024-04-20T21:32:33.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.052+0000 7f01ff400640 1 -- 172.21.15.7:0/511877455 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f01f809c9e0 con 0x7f0208102840 2024-04-20T21:32:33.051 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:32:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 msgr2=0x7f01ec0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 0x7f01ec0a6060 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01f402c190 tx=0x7f01f4000bc0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 msgr2=0x7f02081021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01f800e9f0 tx=0x7f01f800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 shutdown_connections 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f01ec0a3ba0 0x7f01ec0a6060 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0208105170 0x7f02081007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0208107aa0 0x7f0208100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 --2- 172.21.15.7:0/511877455 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0208102840 0x7f02081021b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 >> 172.21.15.7:0/511877455 conn(0x7f02080fc250 msgr2=0x7f0208103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.055+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 shutdown_connections 2024-04-20T21:32:33.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.056+0000 7f0207e00640 1 -- 172.21.15.7:0/511877455 wait complete. 2024-04-20T21:32:33.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 -- 172.21.15.7:0/3582309837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 msgr2=0x7f7f94109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 --2- 172.21.15.7:0/3582309837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f94109b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009900 tx=0x7f7f8002bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 -- 172.21.15.7:0/3582309837 shutdown_connections 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 --2- 172.21.15.7:0/3582309837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f94109b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 --2- 172.21.15.7:0/3582309837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 --2- 172.21.15.7:0/3582309837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f94108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 -- 172.21.15.7:0/3582309837 >> 172.21.15.7:0/3582309837 conn(0x7f7f94077450 msgr2=0x7f7f94079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 -- 172.21.15.7:0/3582309837 shutdown_connections 2024-04-20T21:32:33.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.290+0000 7f7f93200640 1 -- 172.21.15.7:0/3582309837 wait complete. 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.291+0000 7f7f93200640 1 Processor -- start 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.291+0000 7f7f93200640 1 -- start start 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.291+0000 7f7f93200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f9419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.291+0000 7f7f93200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f93200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f93200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94114190 con 0x7f7f9410a060 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f93200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94114010 con 0x7f7f941096a0 2024-04-20T21:32:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f93200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7f94114310 con 0x7f7f94108500 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f9419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53754/0 (socket says 172.21.15.7:53754) 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38150/0 (socket says 172.21.15.7:38150) 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 -- 172.21.15.7:0/643353273 learned_addr learned my addr 172.21.15.7:0/643353273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f9419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43106/0 (socket says 172.21.15.7:43106) 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 -- 172.21.15.7:0/643353273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 msgr2=0x7f7f9419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f9419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 -- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 msgr2=0x7f7f941a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f91400640 1 -- 172.21.15.7:0/643353273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f941a63b0 con 0x7f7f941096a0 2024-04-20T21:32:33.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.292+0000 7f7f92800640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f91400640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f80000c00 tx=0x7f7f80005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80043020 con 0x7f7f941096a0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f800054e0 con 0x7f7f941096a0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80003e60 con 0x7f7f941096a0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941a6640 con 0x7f7f941096a0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.293+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f941a6b40 con 0x7f7f941096a0 2024-04-20T21:32:33.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.294+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f9410f770 con 0x7f7f941096a0 2024-04-20T21:32:33.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.296+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f7f80046020 con 0x7f7f941096a0 2024-04-20T21:32:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.298+0000 7f7f8aa00640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 0x7f7f780a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.298+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7f800f25f0 con 0x7f7f941096a0 2024-04-20T21:32:33.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.298+0000 7f7f91e00640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 0x7f7f780a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.299+0000 7f7f91e00640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 0x7f7f780a6060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005fd0 tx=0x7f7f8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:33.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.302+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7f8004d020 con 0x7f7f941096a0 2024-04-20T21:32:33.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3270902068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/84427689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/511877455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3270902068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/84427689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/511877455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3270902068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/84427689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/511877455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:33.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.551+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f9402d5d0 con 0x7f7f941096a0 2024-04-20T21:32:33.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.553+0000 7f7f8aa00640 1 -- 172.21.15.7:0/643353273 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f7f800befa0 con 0x7f7f941096a0 2024-04-20T21:32:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:33.552 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:32:33.552 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:32:33.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.555+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 msgr2=0x7f7f780a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.555+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 0x7f7f780a6060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005fd0 tx=0x7f7f8400b390 comp rx=0 tx=0).stop 2024-04-20T21:32:33.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.555+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 msgr2=0x7f7f9419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.555+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f80000c00 tx=0x7f7f80005dd0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.555+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 shutdown_connections 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7f780a3ba0 0x7f7f780a6060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f941096a0 0x7f7f9419edc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9410a060 0x7f7f941a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 --2- 172.21.15.7:0/643353273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f94108500 0x7f7f9419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 >> 172.21.15.7:0/643353273 conn(0x7f7f94077450 msgr2=0x7f7f9410c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 shutdown_connections 2024-04-20T21:32:33.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.556+0000 7f7f93200640 1 -- 172.21.15.7:0/643353273 wait complete. 2024-04-20T21:32:33.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 -- 172.21.15.7:0/1366900855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45581040c0 msgr2=0x7f45581044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 --2- 172.21.15.7:0/1366900855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45581040c0 0x7f45581044e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4550009ca0 tx=0x7f455002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 -- 172.21.15.7:0/1366900855 shutdown_connections 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 --2- 172.21.15.7:0/1366900855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4558105c70 0x7f455810c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 --2- 172.21.15.7:0/1366900855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f4558105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 --2- 172.21.15.7:0/1366900855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45581040c0 0x7f45581044e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 -- 172.21.15.7:0/1366900855 >> 172.21.15.7:0/1366900855 conn(0x7f45580ff630 msgr2=0x7f4558101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 -- 172.21.15.7:0/1366900855 shutdown_connections 2024-04-20T21:32:33.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.784+0000 7f455d800640 1 -- 172.21.15.7:0/1366900855 wait complete. 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 Processor -- start 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 -- start start 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 0x7f455819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f455819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f455810fd10 con 0x7f4558105c70 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f455810fb90 con 0x7f45581040c0 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.785+0000 7f455d800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f455810fe90 con 0x7f45581052b0 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f4557400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f455819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53768/0 (socket says 172.21.15.7:53768) 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 -- 172.21.15.7:0/3043648895 learned_addr learned my addr 172.21.15.7:0/3043648895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f4557e00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 0x7f455819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 -- 172.21.15.7:0/3043648895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 msgr2=0x7f455819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f455819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 -- 172.21.15.7:0/3043648895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 msgr2=0x7f455819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 0x7f455819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 -- 172.21.15.7:0/3043648895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45581a1fd0 con 0x7f4558105c70 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f4557e00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 0x7f455819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f4557400640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f455819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.786+0000 7f455ce00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f454800e970 tx=0x7f454800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.787+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454800ccc0 con 0x7f4558105c70 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.787+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454800ce60 con 0x7f4558105c70 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.787+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548010490 con 0x7f4558105c70 2024-04-20T21:32:33.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.787+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45581a22c0 con 0x7f4558105c70 2024-04-20T21:32:33.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.787+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45581a27c0 con 0x7f4558105c70 2024-04-20T21:32:33.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.788+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455810b3e0 con 0x7f4558105c70 2024-04-20T21:32:33.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.790+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f4548005020 con 0x7f4558105c70 2024-04-20T21:32:33.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.792+0000 7f4554c00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 0x7f453c0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:33.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.792+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f45480d0890 con 0x7f4558105c70 2024-04-20T21:32:33.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.792+0000 7f4557e00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 0x7f453c0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:33.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.793+0000 7f4557e00640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 0x7f453c0a6060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f455002c630 tx=0x7f4550009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:33.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:33.796+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f454809d240 con 0x7f4558105c70 2024-04-20T21:32:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.009+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4558062b60 con 0x7f4558105c70 2024-04-20T21:32:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.013+0000 7f4554c00640 1 -- 172.21.15.7:0/3043648895 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7f454809cb40 con 0x7f4558105c70 2024-04-20T21:32:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:34.013 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:32:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.016+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 msgr2=0x7f453c0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.016+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 0x7f453c0a6060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f455002c630 tx=0x7f4550009360 comp rx=0 tx=0).stop 2024-04-20T21:32:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.016+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 msgr2=0x7f45581a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.016+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f454800e970 tx=0x7f454800ed70 comp rx=0 tx=0).stop 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 shutdown_connections 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f453c0a3ba0 0x7f453c0a6060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4558105c70 0x7f45581a1960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45581052b0 0x7f455819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 --2- 172.21.15.7:0/3043648895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45581040c0 0x7f455819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 >> 172.21.15.7:0/3043648895 conn(0x7f45580ff630 msgr2=0x7f45581085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 shutdown_connections 2024-04-20T21:32:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.017+0000 7f455d800640 1 -- 172.21.15.7:0/3043648895 wait complete. 2024-04-20T21:32:34.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:32:34.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 -- 172.21.15.7:0/2299626533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc1085e0 msgr2=0x7f3abc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/2299626533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc1085e0 0x7f3abc108a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009900 tx=0x7f3ab002bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 -- 172.21.15.7:0/2299626533 shutdown_connections 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/2299626533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc10a190 0x7f3abc110a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/2299626533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1097d0 0x7f3abc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/2299626533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc1085e0 0x7f3abc108a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 -- 172.21.15.7:0/2299626533 >> 172.21.15.7:0/2299626533 conn(0x7f3abc0775b0 msgr2=0x7f3abc0799d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 -- 172.21.15.7:0/2299626533 shutdown_connections 2024-04-20T21:32:34.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.249+0000 7f3ac3c00640 1 -- 172.21.15.7:0/2299626533 wait complete. 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 Processor -- start 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 -- start start 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1085e0 0x7f3abc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc114250 con 0x7f3abc10a190 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc1140d0 con 0x7f3abc1097d0 2024-04-20T21:32:34.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.250+0000 7f3ac3c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc1143d0 con 0x7f3abc1085e0 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3ac1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3ac1200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1085e0 0x7f3abc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3ac1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53778/0 (socket says 172.21.15.7:53778) 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3ac1c00640 1 -- 172.21.15.7:0/251468028 learned_addr learned my addr 172.21.15.7:0/251468028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38204/0 (socket says 172.21.15.7:38204) 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 -- 172.21.15.7:0/251468028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1085e0 msgr2=0x7f3abc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1085e0 0x7f3abc19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 -- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 msgr2=0x7f3abc1a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3abbe00640 1 -- 172.21.15.7:0/251468028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc1a6540 con 0x7f3abc1097d0 2024-04-20T21:32:34.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.251+0000 7f3ac1c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:34.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3abbe00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9f0 tx=0x7f3aa800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800ccc0 con 0x7f3abc1097d0 2024-04-20T21:32:34.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc1a6830 con 0x7f3abc1097d0 2024-04-20T21:32:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa8005020 con 0x7f3abc1097d0 2024-04-20T21:32:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80049e0 con 0x7f3abc1097d0 2024-04-20T21:32:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.252+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc1a6d30 con 0x7f3abc1097d0 2024-04-20T21:32:34.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.253+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc10f980 con 0x7f3abc1097d0 2024-04-20T21:32:34.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.256+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3aa8005560 con 0x7f3abc1097d0 2024-04-20T21:32:34.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.257+0000 7f3ab9600640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.257+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3aa80d03b0 con 0x7f3abc1097d0 2024-04-20T21:32:34.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.257+0000 7f3ac1200640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.258+0000 7f3ac1200640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0000c00 tx=0x7f3ab0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.261+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3aa80051d0 con 0x7f3abc1097d0 2024-04-20T21:32:34.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:34 smithi186 ceph-mon[22157]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:32:34.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:34 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/643353273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.367 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:34 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3043648895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:34 smithi016 ceph-mon[21409]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:32:34.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:34 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/643353273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:34 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3043648895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.474+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3abc062b60 con 0x7f3abc1097d0 2024-04-20T21:32:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.475+0000 7f3ab9600640 1 -- 172.21.15.7:0/251468028 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f3aa809cd60 con 0x7f3abc1097d0 2024-04-20T21:32:34.473 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.477+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 msgr2=0x7f3aa40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.477+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0000c00 tx=0x7f3ab0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 msgr2=0x7f3abc19ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800e9f0 tx=0x7f3aa800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 shutdown_connections 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3aa40a3bb0 0x7f3aa40a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3abc1097d0 0x7f3abc19ee60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3abc10a190 0x7f3abc1a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 --2- 172.21.15.7:0/251468028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3abc1085e0 0x7f3abc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 >> 172.21.15.7:0/251468028 conn(0x7f3abc0775b0 msgr2=0x7f3abc10cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 shutdown_connections 2024-04-20T21:32:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.478+0000 7f3ac3c00640 1 -- 172.21.15.7:0/251468028 wait complete. 2024-04-20T21:32:34.493 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:34 smithi007 ceph-mon[16867]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:32:34.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/643353273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.494 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3043648895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:34.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.713+0000 7ffac9200640 1 -- 172.21.15.7:0/4238106449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4101dd0 msgr2=0x7ffac41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 --2- 172.21.15.7:0/4238106449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4101dd0 0x7ffac41021f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009900 tx=0x7ffab402bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 -- 172.21.15.7:0/4238106449 shutdown_connections 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 --2- 172.21.15.7:0/4238106449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4103980 0x7ffac410a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 --2- 172.21.15.7:0/4238106449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 0x7ffac4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 --2- 172.21.15.7:0/4238106449 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4101dd0 0x7ffac41021f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 -- 172.21.15.7:0/4238106449 >> 172.21.15.7:0/4238106449 conn(0x7ffac40fd340 msgr2=0x7ffac40ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 -- 172.21.15.7:0/4238106449 shutdown_connections 2024-04-20T21:32:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.714+0000 7ffac9200640 1 -- 172.21.15.7:0/4238106449 wait complete. 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.715+0000 7ffac9200640 1 Processor -- start 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.715+0000 7ffac9200640 1 -- start start 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 0x7ffac4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 0x7ffac419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410daa0 con 0x7ffac4102fc0 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410d920 con 0x7ffac4101dd0 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac9200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410dc20 con 0x7ffac4103980 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 0x7ffac4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 0x7ffac419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38232/0 (socket says 172.21.15.7:38232) 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3400640 1 -- 172.21.15.7:0/4018096241 learned_addr learned my addr 172.21.15.7:0/4018096241 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3e00640 1 -- 172.21.15.7:0/4018096241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 msgr2=0x7ffac41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.716+0000 7ffac3e00640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac3e00640 1 -- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 msgr2=0x7ffac4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac3e00640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 0x7ffac4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac3e00640 1 -- 172.21.15.7:0/4018096241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac419fd10 con 0x7ffac4103980 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac3400640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac3e00640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 0x7ffac419f6a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e970 tx=0x7ffabc00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00ccc0 con 0x7ffac4103980 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc005020 con 0x7ffac4103980 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac419ffa0 con 0x7ffac4103980 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004a00 con 0x7ffac4103980 2024-04-20T21:32:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.717+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a04c0 con 0x7ffac4103980 2024-04-20T21:32:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.718+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac41090f0 con 0x7ffac4103980 2024-04-20T21:32:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.721+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7ffabc0051d0 con 0x7ffac4103980 2024-04-20T21:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.722+0000 7ffabbe00640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.722+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ffabc0d0420 con 0x7ffac4103980 2024-04-20T21:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.723+0000 7ffac3400640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:34.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.723+0000 7ffac3400640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006010 tx=0x7ffab402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.725+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffabc09ce60 con 0x7ffac4103980 2024-04-20T21:32:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.931+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffac4062b60 con 0x7ffac4103980 2024-04-20T21:32:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.932+0000 7ffabbe00640 1 -- 172.21.15.7:0/4018096241 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ffabc09c760 con 0x7ffac4103980 2024-04-20T21:32:34.931 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 msgr2=0x7ffaa80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4006010 tx=0x7ffab402df90 comp rx=0 tx=0).stop 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 msgr2=0x7ffac419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 0x7ffac419f6a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00e970 tx=0x7ffabc00ee30 comp rx=0 tx=0).stop 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 shutdown_connections 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffaa80a3bb0 0x7ffaa80a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffac4103980 0x7ffac419f6a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffac4102fc0 0x7ffac4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 --2- 172.21.15.7:0/4018096241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffac4101dd0 0x7ffac41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 >> 172.21.15.7:0/4018096241 conn(0x7ffac40fd340 msgr2=0x7ffac41062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 shutdown_connections 2024-04-20T21:32:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:34.935+0000 7ffac9200640 1 -- 172.21.15.7:0/4018096241 wait complete. 2024-04-20T21:32:34.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 -- 172.21.15.7:0/1864769100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103190 msgr2=0x7f4460103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 --2- 172.21.15.7:0/1864769100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103190 0x7f4460103610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4448009900 tx=0x7f444802bc50 comp rx=0 tx=0).stop 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 -- 172.21.15.7:0/1864769100 shutdown_connections 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 --2- 172.21.15.7:0/1864769100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460103b50 0x7f446010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 --2- 172.21.15.7:0/1864769100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103190 0x7f4460103610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 --2- 172.21.15.7:0/1864769100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460101fa0 0x7f44601023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 -- 172.21.15.7:0/1864769100 >> 172.21.15.7:0/1864769100 conn(0x7f44600fd4f0 msgr2=0x7f44600ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 -- 172.21.15.7:0/1864769100 shutdown_connections 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.175+0000 7f4467c00640 1 -- 172.21.15.7:0/1864769100 wait complete. 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.176+0000 7f4467c00640 1 Processor -- start 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.176+0000 7f4467c00640 1 -- start start 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460101fa0 0x7f4460075d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 0x7f4460074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f446010dc80 con 0x7f4460103b50 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f446010db00 con 0x7f4460103190 2024-04-20T21:32:35.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4467c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f446010de00 con 0x7f4460101fa0 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4465c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 0x7f4460074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38260/0 (socket says 172.21.15.7:38260) 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 -- 172.21.15.7:0/244021774 learned_addr learned my addr 172.21.15.7:0/244021774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f4465200640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460101fa0 0x7f4460075d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 -- 172.21.15.7:0/244021774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460101fa0 msgr2=0x7f4460075d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460101fa0 0x7f4460075d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.177+0000 7f445fe00640 1 -- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 msgr2=0x7f4460074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445fe00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 0x7f4460074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445fe00640 1 -- 172.21.15.7:0/244021774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4460074dd0 con 0x7f4460103190 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f4465c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 0x7f4460074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:35.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445fe00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f444802c260 tx=0x7f444802c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448041070 con 0x7f4460103190 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4448004450 con 0x7f4460103190 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4460075060 con 0x7f4460103190 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448034440 con 0x7f4460103190 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.178+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44601a6cd0 con 0x7f4460103190 2024-04-20T21:32:35.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.180+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4460109260 con 0x7f4460103190 2024-04-20T21:32:35.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.181+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f4448044070 con 0x7f4460103190 2024-04-20T21:32:35.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.183+0000 7f445d600640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 0x7f44440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.183+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f44480f1e90 con 0x7f4460103190 2024-04-20T21:32:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.183+0000 7f4465200640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 0x7f44440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.184+0000 7f4465200640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 0x7f44440a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4460102ff0 tx=0x7f4450009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.187+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f44480be1f0 con 0x7f4460103190 2024-04-20T21:32:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:35 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/251468028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:35 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4018096241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.398+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4460062b60 con 0x7f4460103190 2024-04-20T21:32:35.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.399+0000 7f445d600640 1 -- 172.21.15.7:0/244021774 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f44480c24f0 con 0x7f4460103190 2024-04-20T21:32:35.398 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.401+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 msgr2=0x7f44440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.401+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 0x7f44440a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4460102ff0 tx=0x7f4450009500 comp rx=0 tx=0).stop 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.401+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 msgr2=0x7f4460074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.401+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f444802c260 tx=0x7f444802c670 comp rx=0 tx=0).stop 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 shutdown_connections 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f44440a3bb0 0x7f44440a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4460103190 0x7f4460074350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4460103b50 0x7f4460074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 --2- 172.21.15.7:0/244021774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4460101fa0 0x7f4460075d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 >> 172.21.15.7:0/244021774 conn(0x7f44600fd4f0 msgr2=0x7f4460106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 shutdown_connections 2024-04-20T21:32:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.402+0000 7f4467c00640 1 -- 172.21.15.7:0/244021774 wait complete. 2024-04-20T21:32:35.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:35.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:35 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/251468028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:35 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4018096241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:35 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/251468028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:35 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4018096241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:32:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 -- 172.21.15.7:0/989011639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c075fa0 msgr2=0x7f534c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 --2- 172.21.15.7:0/989011639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c075fa0 0x7f534c0743f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f533c009930 tx=0x7f533c02bc20 comp rx=0 tx=0).stop 2024-04-20T21:32:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 -- 172.21.15.7:0/989011639 shutdown_connections 2024-04-20T21:32:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 --2- 172.21.15.7:0/989011639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 0x7f534c110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 --2- 172.21.15.7:0/989011639 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c0749c0 0x7f534c074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 --2- 172.21.15.7:0/989011639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c075fa0 0x7f534c0743f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 -- 172.21.15.7:0/989011639 >> 172.21.15.7:0/989011639 conn(0x7f534c0fd420 msgr2=0x7f534c0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 -- 172.21.15.7:0/989011639 shutdown_connections 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.632+0000 7f534b200640 1 -- 172.21.15.7:0/989011639 wait complete. 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 Processor -- start 2024-04-20T21:32:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 -- start start 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c075fa0 0x7f534c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 0x7f534c1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f534c114120 con 0x7f534c10e460 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f534c113fa0 con 0x7f534c0749c0 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.634+0000 7f534b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1142a0 con 0x7f534c075fa0 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f534a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 0x7f534c1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38278/0 (socket says 172.21.15.7:38278) 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 -- 172.21.15.7:0/3977130343 learned_addr learned my addr 172.21.15.7:0/3977130343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349400640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c075fa0 0x7f534c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 -- 172.21.15.7:0/3977130343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c075fa0 msgr2=0x7f534c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c075fa0 0x7f534c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 -- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 msgr2=0x7f534c1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 0x7f534c1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.635+0000 7f5349e00640 1 -- 172.21.15.7:0/3977130343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c1a63f0 con 0x7f534c0749c0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f5349e00640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f533c02c7a0 tx=0x7f533c02c5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c041070 con 0x7f534c0749c0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533c004450 con 0x7f534c0749c0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c034440 con 0x7f534c0749c0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1a6680 con 0x7f534c0749c0 2024-04-20T21:32:35.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.636+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a6b20 con 0x7f534c0749c0 2024-04-20T21:32:35.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.637+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c104630 con 0x7f534c0749c0 2024-04-20T21:32:35.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.639+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f533c044070 con 0x7f534c0749c0 2024-04-20T21:32:35.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.641+0000 7f5342a00640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 0x7f53300a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:35.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.641+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f533c0f2310 con 0x7f534c0749c0 2024-04-20T21:32:35.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.641+0000 7f5349400640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 0x7f53300a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:35.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.642+0000 7f5349400640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 0x7f53300a60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f534c19fce0 tx=0x7f5338005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:35.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.643+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f533c0becc0 con 0x7f534c0749c0 2024-04-20T21:32:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.893+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f534c02d5d0 con 0x7f534c0749c0 2024-04-20T21:32:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.895+0000 7f5342a00640 1 -- 172.21.15.7:0/3977130343 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f533c0be5c0 con 0x7f534c0749c0 2024-04-20T21:32:35.894 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:35.894 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:32:35.894 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 msgr2=0x7f53300a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 0x7f53300a60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f534c19fce0 tx=0x7f5338005f90 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 msgr2=0x7f534c19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f533c02c7a0 tx=0x7f533c02c5c0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 shutdown_connections 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f53300a3c20 0x7f53300a60e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c10e460 0x7f534c1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f534c075fa0 0x7f534c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 --2- 172.21.15.7:0/3977130343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f534c0749c0 0x7f534c19e7d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 >> 172.21.15.7:0/3977130343 conn(0x7f534c0fd420 msgr2=0x7f534c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:35.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 shutdown_connections 2024-04-20T21:32:35.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:35.898+0000 7f534b200640 1 -- 172.21.15.7:0/3977130343 wait complete. 2024-04-20T21:32:35.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 -- 172.21.15.7:0/2186325184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c105de0 msgr2=0x7f889c10c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 --2- 172.21.15.7:0/2186325184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c105de0 0x7f889c10c670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8894009ca0 tx=0x7f889402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 -- 172.21.15.7:0/2186325184 shutdown_connections 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 --2- 172.21.15.7:0/2186325184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c105de0 0x7f889c10c670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 --2- 172.21.15.7:0/2186325184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105420 0x7f889c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 --2- 172.21.15.7:0/2186325184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c104230 0x7f889c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 -- 172.21.15.7:0/2186325184 >> 172.21.15.7:0/2186325184 conn(0x7f889c0ff7e0 msgr2=0x7f889c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 -- 172.21.15.7:0/2186325184 shutdown_connections 2024-04-20T21:32:36.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.126+0000 7f88a3600640 1 -- 172.21.15.7:0/2186325184 wait complete. 2024-04-20T21:32:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 Processor -- start 2024-04-20T21:32:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 -- start start 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c104230 0x7f889c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 0x7f889c1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10fe50 con 0x7f889c105420 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10fcd0 con 0x7f889c104230 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.127+0000 7f88a3600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10ffd0 con 0x7f889c105de0 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53850/0 (socket says 172.21.15.7:53850) 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 -- 172.21.15.7:0/1648788039 learned_addr learned my addr 172.21.15.7:0/1648788039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f88a1600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 0x7f889c1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f88a0c00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c104230 0x7f889c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 -- 172.21.15.7:0/1648788039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 msgr2=0x7f889c1a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 0x7f889c1a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 -- 172.21.15.7:0/1648788039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c104230 msgr2=0x7f889c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c104230 0x7f889c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 -- 172.21.15.7:0/1648788039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c1a2060 con 0x7f889c105420 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f88a1600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 0x7f889c1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.128+0000 7f889be00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f888800e9f0 tx=0x7f888800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.129+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888800cd80 con 0x7f889c105420 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.129+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c1a2350 con 0x7f889c105420 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.129+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8888005500 con 0x7f889c105420 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.129+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888010690 con 0x7f889c105420 2024-04-20T21:32:36.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.129+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1a28a0 con 0x7f889c105420 2024-04-20T21:32:36.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.130+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889c10b4f0 con 0x7f889c105420 2024-04-20T21:32:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.133+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f88880046c0 con 0x7f889c105420 2024-04-20T21:32:36.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.134+0000 7f8899600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 0x7f88840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.134+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f88880d0750 con 0x7f889c105420 2024-04-20T21:32:36.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.134+0000 7f88a0c00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 0x7f88840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.135+0000 7f88a0c00640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 0x7f88840a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f889c105280 tx=0x7f8890005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:36.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.138+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f888809d160 con 0x7f889c105420 2024-04-20T21:32:36.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:36 smithi007 ceph-mon[16867]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:32:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/244021774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:36.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:36.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.345+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f889c062b60 con 0x7f889c105420 2024-04-20T21:32:36.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.349+0000 7f8899600640 1 -- 172.21.15.7:0/1648788039 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7f888809ca60 con 0x7f889c105420 2024-04-20T21:32:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:36.348 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 msgr2=0x7f88840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 0x7f88840a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f889c105280 tx=0x7f8890005f90 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 msgr2=0x7f889c19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f888800e9f0 tx=0x7f888800edf0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 shutdown_connections 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f88840a3c30 0x7f88840a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f889c105de0 0x7f889c1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f889c105420 0x7f889c19a980 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 --2- 172.21.15.7:0/1648788039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c104230 0x7f889c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 >> 172.21.15.7:0/1648788039 conn(0x7f889c0ff7e0 msgr2=0x7f889c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:36.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 shutdown_connections 2024-04-20T21:32:36.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.352+0000 7f88a3600640 1 -- 172.21.15.7:0/1648788039 wait complete. 2024-04-20T21:32:36.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:32:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:36 smithi016 ceph-mon[21409]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:32:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:36 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/244021774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:36.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:36 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:36 smithi186 ceph-mon[22157]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:32:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:36 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/244021774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:32:36.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:36 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3977130343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 -- 172.21.15.7:0/2019879109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec105250 msgr2=0x7f41ec107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 --2- 172.21.15.7:0/2019879109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec105250 0x7f41ec107640 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009900 tx=0x7f41d402bc30 comp rx=0 tx=0).stop 2024-04-20T21:32:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 -- 172.21.15.7:0/2019879109 shutdown_connections 2024-04-20T21:32:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 --2- 172.21.15.7:0/2019879109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 0x7f41ec109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 --2- 172.21.15.7:0/2019879109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec105250 0x7f41ec107640 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 --2- 172.21.15.7:0/2019879109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec102920 0x7f41ec104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 -- 172.21.15.7:0/2019879109 >> 172.21.15.7:0/2019879109 conn(0x7f41ec0fc330 msgr2=0x7f41ec0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 -- 172.21.15.7:0/2019879109 shutdown_connections 2024-04-20T21:32:36.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.584+0000 7f41f1c00640 1 -- 172.21.15.7:0/2019879109 wait complete. 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.585+0000 7f41f1c00640 1 Processor -- start 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.585+0000 7f41f1c00640 1 -- start start 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec105250 0x7f41ec19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 0x7f41ec1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10db70 con 0x7f41ec102920 2024-04-20T21:32:36.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10d9f0 con 0x7f41ec105250 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10dcf0 con 0x7f41ec107b80 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53860/0 (socket says 172.21.15.7:53860) 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41ebe00640 1 -- 172.21.15.7:0/4183393109 learned_addr learned my addr 172.21.15.7:0/4183393109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41f1200640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 0x7f41ec1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.586+0000 7f41eb400640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec105250 0x7f41ec19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 -- 172.21.15.7:0/4183393109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 msgr2=0x7f41ec1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 0x7f41ec1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 -- 172.21.15.7:0/4183393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec105250 msgr2=0x7f41ec19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec105250 0x7f41ec19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 -- 172.21.15.7:0/4183393109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec1a2020 con 0x7f41ec102920 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41ebe00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00db80 tx=0x7f41dc00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:36.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc017070 con 0x7f41ec102920 2024-04-20T21:32:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41dc003b70 con 0x7f41ec102920 2024-04-20T21:32:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0106c0 con 0x7f41ec102920 2024-04-20T21:32:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec1a2310 con 0x7f41ec102920 2024-04-20T21:32:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.587+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1a2830 con 0x7f41ec102920 2024-04-20T21:32:36.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.589+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec1096d0 con 0x7f41ec102920 2024-04-20T21:32:36.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.591+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f41dc004020 con 0x7f41ec102920 2024-04-20T21:32:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.593+0000 7f41e8c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 0x7f41d00a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.593+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f41dc0d01f0 con 0x7f41ec102920 2024-04-20T21:32:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.593+0000 7f41eb400640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 0x7f41d00a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:36.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.594+0000 7f41eb400640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 0x7f41d00a6060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41d402c170 tx=0x7f41d402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:36.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.598+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f41dc020070 con 0x7f41ec102920 2024-04-20T21:32:36.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.846+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41ec02d5d0 con 0x7f41ec102920 2024-04-20T21:32:36.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.848+0000 7f41e8c00640 1 -- 172.21.15.7:0/4183393109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f41dc09cba0 con 0x7f41ec102920 2024-04-20T21:32:36.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:36.847 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:32:36.847 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:32:36.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 msgr2=0x7f41d00a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 0x7f41d00a6060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f41d402c170 tx=0x7f41d402df90 comp rx=0 tx=0).stop 2024-04-20T21:32:36.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 msgr2=0x7f41ec19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:36.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00db80 tx=0x7f41dc00b630 comp rx=0 tx=0).stop 2024-04-20T21:32:36.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 shutdown_connections 2024-04-20T21:32:36.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f41d00a3ba0 0x7f41d00a6060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41ec105250 0x7f41ec19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec107b80 0x7f41ec1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 --2- 172.21.15.7:0/4183393109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41ec102920 0x7f41ec19a400 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:36.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 >> 172.21.15.7:0/4183393109 conn(0x7f41ec0fc330 msgr2=0x7f41ec101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:36.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 shutdown_connections 2024-04-20T21:32:36.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:36.851+0000 7f41f1c00640 1 -- 172.21.15.7:0/4183393109 wait complete. 2024-04-20T21:32:36.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T21:30:52.886616+0000', 'modified': '2024-04-20T21:30:58.437063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24559, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/2330254135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2330254135}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2330254135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/4051126294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 4051126294}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 4051126294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/2162825015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2162825015}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 2162825015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.186:6837/2315081064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 2315081064}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 2315081064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6839/2448649165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2448649165}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2448649165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/518118418', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 518118418}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 518118418}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24575, 'qdb_cluster': [24535, 24559, 24575]} 2024-04-20T21:32:36.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-20T21:32:36.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 -- 172.21.15.7:0/2896423041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c1095d0 msgr2=0x7f078c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 --2- 172.21.15.7:0/2896423041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c1095d0 0x7f078c109a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0778009900 tx=0x7f077802bc40 comp rx=0 tx=0).stop 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 -- 172.21.15.7:0/2896423041 shutdown_connections 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 --2- 172.21.15.7:0/2896423041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c109f90 0x7f078c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 --2- 172.21.15.7:0/2896423041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c1095d0 0x7f078c109a50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 --2- 172.21.15.7:0/2896423041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 -- 172.21.15.7:0/2896423041 >> 172.21.15.7:0/2896423041 conn(0x7f078c0773a0 msgr2=0x7f078c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 -- 172.21.15.7:0/2896423041 shutdown_connections 2024-04-20T21:32:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.079+0000 7f0792c00640 1 -- 172.21.15.7:0/2896423041 wait complete. 2024-04-20T21:32:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.080+0000 7f0792c00640 1 Processor -- start 2024-04-20T21:32:37.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 -- start start 2024-04-20T21:32:37.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c1095d0 0x7f078c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 0x7f078c1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f078c1140c0 con 0x7f078c109f90 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f078c113f40 con 0x7f078c1095d0 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0792c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f078c114240 con 0x7f078c1083e0 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.081+0000 7f0791800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0791800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43230/0 (socket says 172.21.15.7:43230) 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0791800640 1 -- 172.21.15.7:0/1857551184 learned_addr learned my addr 172.21.15.7:0/1857551184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 0x7f078c1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0790e00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c1095d0 0x7f078c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 -- 172.21.15.7:0/1857551184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 msgr2=0x7f078c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 -- 172.21.15.7:0/1857551184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c1095d0 msgr2=0x7f078c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c1095d0 0x7f078c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 -- 172.21.15.7:0/1857551184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c1a62f0 con 0x7f078c109f90 2024-04-20T21:32:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0791800640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.082+0000 7f0792200640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 0x7f078c1a5c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f078400ec50 tx=0x7f078400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.083+0000 7f078a000640 1 -- 172.21.15.7:0/1857551184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784003b70 con 0x7f078c109f90 2024-04-20T21:32:37.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.083+0000 7f078a000640 1 -- 172.21.15.7:0/1857551184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0784003d10 con 0x7f078c109f90 2024-04-20T21:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.083+0000 7f078a000640 1 -- 172.21.15.7:0/1857551184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784010890 con 0x7f078c109f90 2024-04-20T21:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.083+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078c1a65e0 con 0x7f078c109f90 2024-04-20T21:32:37.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.083+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078c1a6b00 con 0x7f078c109f90 2024-04-20T21:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.086+0000 7f0796df3740 1 Processor -- start 2024-04-20T21:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f078a000640 1 -- 172.21.15.7:0/1857551184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0784003fc0 con 0x7f078c109f90 2024-04-20T21:32:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 -- start start 2024-04-20T21:32:37.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2fb40 0x55cad1e2ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad1d969d0 con 0x55cad1e356a0 2024-04-20T21:32:37.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad1d3d270 con 0x55cad1e30530 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.087+0000 7f0796df3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad1d92990 con 0x55cad1e2fb40 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0790e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0790e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53880/0 (socket says 172.21.15.7:53880) 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0790e00640 1 -- 172.21.15.7:0/2509114223 learned_addr learned my addr 172.21.15.7:0/2509114223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0790e00640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2fb40 0x55cad1e2ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 -- 172.21.15.7:0/2509114223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2fb40 msgr2=0x55cad1e2ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2fb40 0x55cad1e2ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 -- 172.21.15.7:0/2509114223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 msgr2=0x55cad1e37b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0791800640 1 -- 172.21.15.7:0/2509114223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad1aa92d0 con 0x55cad1e30530 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.088+0000 7f0790e00640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f0791800640 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e35030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07800114c0 tx=0x7f0780011980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d2bb8c34e2fd296 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f078a000640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 0x7f07740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f078a000640 1 -- 172.21.15.7:0/1857551184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f07840d0900 con 0x7f078c109f90 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f0790e00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 0x7f07740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f077ea00640 1 -- 172.21.15.7:0/2509114223 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780011250 con 0x55cad1e30530 2024-04-20T21:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f077ea00640 1 -- 172.21.15.7:0/2509114223 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0780005500 con 0x55cad1e30530 2024-04-20T21:32:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.089+0000 7f077ea00640 1 -- 172.21.15.7:0/2509114223 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078000f820 con 0x55cad1e30530 2024-04-20T21:32:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- 172.21.15.7:0/2509114223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 msgr2=0x55cad1e35030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e35030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07800114c0 tx=0x7f0780011980 comp rx=0 tx=0).stop 2024-04-20T21:32:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0790e00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 0x7f07740a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f077802c8f0 tx=0x7f0778043040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- 172.21.15.7:0/2509114223 shutdown_connections 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e35030 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e356a0 0x55cad1e37b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 --2- 172.21.15.7:0/2509114223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2fb40 0x55cad1e2ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- 172.21.15.7:0/2509114223 >> 172.21.15.7:0/2509114223 conn(0x55cad1e2b0b0 msgr2=0x55cad1e2b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- 172.21.15.7:0/2509114223 shutdown_connections 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- 172.21.15.7:0/2509114223 wait complete. 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.090+0000 7f0796df3740 1 -- start start 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 Processor -- start 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e3a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e356a0 0x55cad1e415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e38650 con 0x55cad1e2fb40 2024-04-20T21:32:37.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad1d505e0 con 0x55cad1e356a0 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e387d0 con 0x55cad1e30530 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0796df3740 10 client.0 _ll_register_callbacks cb 0x55cad1aa8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0791800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0791800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53886/0 (socket says 172.21.15.7:53886) 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0791800640 1 -- 172.21.15.7:0/677704909 learned_addr learned my addr 172.21.15.7:0/677704909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0790e00640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e3a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.091+0000 7f0792200640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e356a0 0x55cad1e415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 -- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 msgr2=0x55cad1e3a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e3a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 -- 172.21.15.7:0/677704909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e356a0 msgr2=0x55cad1e415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e356a0 0x55cad1e415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 -- 172.21.15.7:0/677704909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad1b736b0 con 0x55cad1e2fb40 2024-04-20T21:32:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0790e00640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e3a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0791800640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0780009350 tx=0x7f0780004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507a6170e33e56d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0773400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780027030 con 0x55cad1e2fb40 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f0780004eb0 con 0x55cad1e2fb40 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.092+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0780005500 con 0x55cad1e2fb40 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f078001d040 con 0x55cad1e2fb40 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 10 client.14877 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cad1e44940 con 0x55cad1e2fb40 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0780005500 con 0x55cad1e2fb40 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f078002e4c0 con 0x55cad1e2fb40 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 10 client.14877 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 10 client.14877 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 20 client.14877 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e46d20 0x55cad1e48190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 4 client.14877 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.093+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad1da43f0 con 0x55cad1e46d20 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.094+0000 7f0790e00640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e46d20 0x55cad1e48190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.094+0000 7f0790e00640 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e46d20 0x55cad1e48190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b178c232184a7baf server_cookie=6ce41dec35422228 in_seq=0 out_seq=0 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.094+0000 7f0773400640 10 client.14877 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.095+0000 7f0773400640 1 -- 172.21.15.7:0/677704909 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cad1da43f0 con 0x55cad1e46d20 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.095+0000 7f0773400640 10 client.14877 handle_command_reply: tid=0 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.095+0000 7f0796df3740 1 client.14877 shutdown 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 msgr2=0x55cad1e3a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0780009350 tx=0x7f0780004950 comp rx=0 tx=0).stop 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 shutdown_connections 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e46d20 0x55cad1e48190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e356a0 0x55cad1e415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e3a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 --2- 172.21.15.7:0/677704909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3a050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 >> 172.21.15.7:0/677704909 conn(0x55cad1e2b0b0 msgr2=0x55cad1e2a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0773400640 0 client.14877 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:32:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 shutdown_connections 2024-04-20T21:32:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 1 -- 172.21.15.7:0/677704909 wait complete. 2024-04-20T21:32:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.096+0000 7f0796df3740 20 client.14877 trim_cache size 0 max 16384 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.115+0000 7f0796df3740 1 Processor -- start 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.115+0000 7f0796df3740 1 -- start start 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e41590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e26280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e464d0 con 0x55cad1e2fb40 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e46ba0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0796df3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e47270 con 0x55cad1e30530 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38364/0 (socket says 172.21.15.7:38364) 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 -- 172.21.15.7:0/1303286416 learned_addr learned my addr 172.21.15.7:0/1303286416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0790e00640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e41590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0792200640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e26280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 -- 172.21.15.7:0/1303286416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 msgr2=0x55cad1e26280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e26280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 -- 172.21.15.7:0/1303286416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 msgr2=0x55cad1e41590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e41590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0791800640 1 -- 172.21.15.7:0/1303286416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad1e2be20 con 0x55cad1e2b0b0 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.116+0000 7f0792200640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e26280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:37.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.117+0000 7f0791800640 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07800114c0 tx=0x7f0780005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab5af5c91413a40f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.117+0000 7f077e000640 1 -- 172.21.15.7:0/1303286416 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078003a070 con 0x55cad1e2b0b0 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.117+0000 7f077e000640 1 -- 172.21.15.7:0/1303286416 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078002ed50 con 0x55cad1e2b0b0 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.117+0000 7f077e000640 1 -- 172.21.15.7:0/1303286416 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780027040 con 0x55cad1e2b0b0 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- 172.21.15.7:0/1303286416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 msgr2=0x55cad1e45d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f07800114c0 tx=0x7f0780005fd0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- 172.21.15.7:0/1303286416 shutdown_connections 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e30530 0x55cad1e26280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e41590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 --2- 172.21.15.7:0/1303286416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e2b0b0 0x55cad1e45d50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- 172.21.15.7:0/1303286416 >> 172.21.15.7:0/1303286416 conn(0x55cad1e31310 msgr2=0x55cad1e2eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- 172.21.15.7:0/1303286416 shutdown_connections 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- 172.21.15.7:0/1303286416 wait complete. 2024-04-20T21:32:37.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 -- start start 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.118+0000 7f0796df3740 1 Processor -- start 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 0x55cad1e3cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e48010 con 0x55cad1e2fb40 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e47940 con 0x55cad1e30530 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad1e486e0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0796df3740 10 client.0 _ll_register_callbacks cb 0x7f0780005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0790e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0792200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0791800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 0x55cad1e3cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0790e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2024-04-20T21:32:37.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0792200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38374/0 (socket says 172.21.15.7:38374) 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.119+0000 7f0790e00640 1 -- 172.21.15.7:0/2163104756 learned_addr learned my addr 172.21.15.7:0/2163104756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 -- 172.21.15.7:0/2163104756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 msgr2=0x55cad1e400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 -- 172.21.15.7:0/2163104756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 msgr2=0x55cad1e3d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 -- 172.21.15.7:0/2163104756 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad1e406a0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0791800640 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 0x55cad1e3cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55cad1e47910 tx=0x7f078002ef00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b3ad0e3580ea48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.120+0000 7f0773400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:32:37.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.123+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780027030 con 0x55cad1e2b0b0 2024-04-20T21:32:37.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.123+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f078002e0b0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.123+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078002ed50 con 0x55cad1e2b0b0 2024-04-20T21:32:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.126+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f078003a070 con 0x55cad1e2b0b0 2024-04-20T21:32:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.126+0000 7f0796df3740 10 client.25000 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.126+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cad1e419a0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.126+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f078002ed50 con 0x55cad1e2b0b0 2024-04-20T21:32:37.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f0780003ab0 con 0x55cad1e2b0b0 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 10 client.25000 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 10 client.25000 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 20 client.25000 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e356a0 0x55cad1eb0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 4 client.25000 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:32:37.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "status"}) -- 0x55cad1d9e260 con 0x55cad1e356a0 2024-04-20T21:32:37.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.127+0000 7f0790e00640 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e356a0 0x55cad1eb0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0790e00640 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e356a0 0x55cad1eb0290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ddb6eeb5b30b9686 server_cookie=f93e74c481d22019 in_seq=0 out_seq=0 2024-04-20T21:32:37.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0773400640 10 client.25000 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:32:37.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0773400640 1 -- 172.21.15.7:0/2163104756 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55cad1d9e260 con 0x55cad1e356a0 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0773400640 10 client.25000 handle_command_reply: tid=0 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 client.25000 shutdown 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 msgr2=0x55cad1e3cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 0x55cad1e3cc30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55cad1e47910 tx=0x7f078002ef00 comp rx=0 tx=0).stop 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 shutdown_connections 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cad1e356a0 0x55cad1eb0290 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad1e30530 0x55cad1e400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0773400640 0 client.25000 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad1e2fb40 0x55cad1e3d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 --2- 172.21.15.7:0/2163104756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad1e2b0b0 0x55cad1e3cc30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 >> 172.21.15.7:0/2163104756 conn(0x55cad1e31310 msgr2=0x55cad1e2eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 shutdown_connections 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 1 -- 172.21.15.7:0/2163104756 wait complete. 2024-04-20T21:32:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.128+0000 7f0796df3740 20 client.25000 trim_cache size 0 max 16384 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24535, 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 99.179653746, 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:32:37.129 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:32:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:32:37.130 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 127.793514028 2024-04-20T21:32:37.130 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 msgr2=0x7f07740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 0x7f07740a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f077802c8f0 tx=0x7f0778043040 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 msgr2=0x7f078c1a5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 0x7f078c1a5c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f078400ec50 tx=0x7f078400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 shutdown_connections 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f07740a3c30 0x7f07740a60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f078c1095d0 0x7f078c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f078c109f90 0x7f078c1a5c80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 --2- 172.21.15.7:0/1857551184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f078c1083e0 0x7f078c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 >> 172.21.15.7:0/1857551184 conn(0x7f078c0773a0 msgr2=0x7f078c10c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.129+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 shutdown_connections 2024-04-20T21:32:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.130+0000 7f0792c00640 1 -- 172.21.15.7:0/1857551184 wait complete. 2024-04-20T21:32:37.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-20T21:32:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 -- 172.21.15.7:0/1962967723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00681083e0 msgr2=0x7f0068108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 --2- 172.21.15.7:0/1962967723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00681083e0 0x7f0068108800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f005c009900 tx=0x7f005c02bc40 comp rx=0 tx=0).stop 2024-04-20T21:32:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 -- 172.21.15.7:0/1962967723 shutdown_connections 2024-04-20T21:32:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 --2- 172.21.15.7:0/1962967723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0068109f90 0x7f0068110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 --2- 172.21.15.7:0/1962967723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 0x7f0068109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 --2- 172.21.15.7:0/1962967723 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00681083e0 0x7f0068108800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 -- 172.21.15.7:0/1962967723 >> 172.21.15.7:0/1962967723 conn(0x7f00680773a0 msgr2=0x7f00680797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 -- 172.21.15.7:0/1962967723 shutdown_connections 2024-04-20T21:32:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.340+0000 7f006f000640 1 -- 172.21.15.7:0/1962967723 wait complete. 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.341+0000 7f006f000640 1 Processor -- start 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.341+0000 7f006f000640 1 -- start start 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 0x7f006819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0068109f90 0x7f00681a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0068114040 con 0x7f00681095d0 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0068113ec0 con 0x7f00681083e0 2024-04-20T21:32:37.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006f000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f00681141c0 con 0x7f0068109f90 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 0x7f006819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38396/0 (socket says 172.21.15.7:38396) 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 -- 172.21.15.7:0/3283609390 learned_addr learned my addr 172.21.15.7:0/3283609390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006e600640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0068109f90 0x7f00681a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 -- 172.21.15.7:0/3283609390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0068109f90 msgr2=0x7f00681a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0068109f90 0x7f00681a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 -- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 msgr2=0x7f006819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 0x7f006819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.342+0000 7f006dc00640 1 -- 172.21.15.7:0/3283609390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00681a6370 con 0x7f00681083e0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f006dc00640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f0066000640 1 -- 172.21.15.7:0/3283609390 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c041070 con 0x7f00681083e0 2024-04-20T21:32:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f0066000640 1 -- 172.21.15.7:0/3283609390 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f005c0044d0 con 0x7f00681083e0 2024-04-20T21:32:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00681a6600 con 0x7f00681083e0 2024-04-20T21:32:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f0066000640 1 -- 172.21.15.7:0/3283609390 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c034440 con 0x7f00681083e0 2024-04-20T21:32:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.343+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681a6af0 con 0x7f00681083e0 2024-04-20T21:32:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.346+0000 7f00733ae740 1 Processor -- start 2024-04-20T21:32:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.346+0000 7f0066000640 1 -- 172.21.15.7:0/3283609390 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f005c044070 con 0x7f00681083e0 2024-04-20T21:32:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.346+0000 7f00733ae740 1 -- start start 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bfd84c9d0 con 0x557bfd8e6530 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557bfd7f3270 con 0x557bfd8eb6a0 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f00733ae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557bfd848990 con 0x557bfd8e5b40 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43294/0 (socket says 172.21.15.7:43294) 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 -- 172.21.15.7:0/3757716894 learned_addr learned my addr 172.21.15.7:0/3757716894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 -- 172.21.15.7:0/3757716894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 msgr2=0x557bfd8edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8edb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 -- 172.21.15.7:0/3757716894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 msgr2=0x557bfd8eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.347+0000 7f006e600640 1 -- 172.21.15.7:0/3757716894 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfd55f2d0 con 0x557bfd8e5b40 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f006e600640 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0060010370 tx=0x7f0060010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1acea0ed7d712d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f005aa00640 1 -- 172.21.15.7:0/3757716894 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060016070 con 0x557bfd8e5b40 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f005aa00640 1 -- 172.21.15.7:0/3757716894 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060004d20 con 0x557bfd8e5b40 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f0066000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 0x7f00500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f005aa00640 1 -- 172.21.15.7:0/3757716894 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060005440 con 0x557bfd8e5b40 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f0066000640 1 -- 172.21.15.7:0/3283609390 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f005c0f1f00 con 0x7f00681083e0 2024-04-20T21:32:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.348+0000 7f006dc00640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 0x7f00500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- 172.21.15.7:0/3757716894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 msgr2=0x557bfd8e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0060010370 tx=0x7f0060010830 comp rx=0 tx=0).stop 2024-04-20T21:32:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- 172.21.15.7:0/3757716894 shutdown_connections 2024-04-20T21:32:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 --2- 172.21.15.7:0/3757716894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8e5f60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- 172.21.15.7:0/3757716894 >> 172.21.15.7:0/3757716894 conn(0x557bfd8e10b0 msgr2=0x557bfd8e14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- 172.21.15.7:0/3757716894 shutdown_connections 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- 172.21.15.7:0/3757716894 wait complete. 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f006dc00640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 0x7f00500a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f005c0ba6c0 tx=0x7f005c0f5940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.349+0000 7f00733ae740 1 -- start start 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 Processor -- start 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8ee650 con 0x557bfd8e6530 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8065e0 con 0x557bfd8eb6a0 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8ee7d0 con 0x557bfd8e5b40 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f00733ae740 10 client.0 _ll_register_callbacks cb 0x557bfd55eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f006d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.350+0000 7f006d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53930/0 (socket says 172.21.15.7:53930) 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 -- 172.21.15.7:0/2403790689 learned_addr learned my addr 172.21.15.7:0/2403790689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006e600640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006e600640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 -- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 msgr2=0x557bfd8f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 -- 172.21.15.7:0/2403790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 msgr2=0x557bfd8f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 -- 172.21.15.7:0/2403790689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfd6296b0 con 0x557bfd8e6530 2024-04-20T21:32:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f006d200640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0054012700 tx=0x7f0054012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd288088277e744d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f004f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005400f8c0 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f005400fa60 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.351+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060004d20 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0054023070 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 10 client.14895 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557bfd8faa80 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0060004d20 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f0054010430 con 0x557bfd8e6530 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 10 client.14895 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 10 client.14895 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 20 client.14895 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd8fd130 0x557bfd8fe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 4 client.14895 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.352+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bfd85a3f0 con 0x557bfd8fd130 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.353+0000 7f006e600640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd8fd130 0x557bfd8fe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.353+0000 7f006e600640 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd8fd130 0x557bfd8fe5a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8c8aeaef366291ca server_cookie=b0ceb070e00b9637 in_seq=0 out_seq=0 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.353+0000 7f004f400640 10 client.14895 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.354+0000 7f004f400640 1 -- 172.21.15.7:0/2403790689 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bfd85a3f0 con 0x557bfd8fd130 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.354+0000 7f004f400640 10 client.14895 handle_command_reply: tid=0 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.354+0000 7f00733ae740 1 client.14895 shutdown 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.354+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 msgr2=0x557bfd8f0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.354+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0054012700 tx=0x7f0054012bc0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 shutdown_connections 2024-04-20T21:32:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd8fd130 0x557bfd8fe5a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f0600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f004f400640 0 client.14895 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 --2- 172.21.15.7:0/2403790689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 >> 172.21.15.7:0/2403790689 conn(0x557bfd8e10b0 msgr2=0x557bfd8e03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 shutdown_connections 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 1 -- 172.21.15.7:0/2403790689 wait complete. 2024-04-20T21:32:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.355+0000 7f00733ae740 20 client.14895 trim_cache size 0 max 16384 2024-04-20T21:32:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 Processor -- start 2024-04-20T21:32:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 -- start start 2024-04-20T21:32:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8ee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8fc8e0 con 0x557bfd8e6530 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8fcfb0 con 0x557bfd8eb6a0 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f00733ae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8fd680 con 0x557bfd8e5b40 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f006d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f006d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53940/0 (socket says 172.21.15.7:53940) 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.375+0000 7f006d200640 1 -- 172.21.15.7:0/61943362 learned_addr learned my addr 172.21.15.7:0/61943362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006e600640 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8ee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006e600640 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 -- 172.21.15.7:0/61943362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 msgr2=0x557bfd8ee530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8ee530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 -- 172.21.15.7:0/61943362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 msgr2=0x557bfd8e1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8e1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 -- 172.21.15.7:0/61943362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfd8ead10 con 0x557bfd8e6530 2024-04-20T21:32:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f006d200640 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00540059f0 tx=0x7f00540123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2daa18c93e1635b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f005a000640 1 -- 172.21.15.7:0/61943362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054010430 con 0x557bfd8e6530 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f005a000640 1 -- 172.21.15.7:0/61943362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00540105d0 con 0x557bfd8e6530 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.376+0000 7f005a000640 1 -- 172.21.15.7:0/61943362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005400fe40 con 0x557bfd8e6530 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 -- 172.21.15.7:0/61943362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 msgr2=0x557bfd8f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f00540059f0 tx=0x7f00540123a0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 -- 172.21.15.7:0/61943362 shutdown_connections 2024-04-20T21:32:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8e1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e6530 0x557bfd8f7600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 --2- 172.21.15.7:0/61943362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e5b40 0x557bfd8ee530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 -- 172.21.15.7:0/61943362 >> 172.21.15.7:0/61943362 conn(0x557bfd8e10b0 msgr2=0x557bfd8dc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 -- 172.21.15.7:0/61943362 shutdown_connections 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.377+0000 7f00733ae740 1 -- 172.21.15.7:0/61943362 wait complete. 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 -- start start 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 Processor -- start 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e6530 0x557bfd8f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8fe420 con 0x557bfd8e5b40 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8fdd50 con 0x557bfd8eb6a0 2024-04-20T21:32:37.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557bfd8feaf0 con 0x557bfd8e6530 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f00733ae740 10 client.0 _ll_register_callbacks cb 0x7f0054009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f006e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f006e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53952/0 (socket says 172.21.15.7:53952) 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.378+0000 7f006e600640 1 -- 172.21.15.7:0/3201131334 learned_addr learned my addr 172.21.15.7:0/3201131334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006d200640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e6530 0x557bfd8f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 -- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e6530 msgr2=0x557bfd8f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e6530 0x557bfd8f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 -- 172.21.15.7:0/3201131334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 msgr2=0x557bfd8f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 -- 172.21.15.7:0/3201131334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfd8f6a50 con 0x557bfd8e5b40 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f006e600640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0060009f40 tx=0x7f0060000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0ef88ee387d781 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f004f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060016040 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00540105d0 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.379+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f0060003970 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0060023040 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 10 client.14901 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557bfd963340 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00540105d0 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f006000f040 con 0x557bfd8e5b40 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 10 client.14901 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 10 client.14901 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 20 client.14901 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd967d90 0x557bfd9641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 4 client.14901 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:32:37.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.380+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "status"}) -- 0x557bfd854260 con 0x557bfd967d90 2024-04-20T21:32:37.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f006d200640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd967d90 0x557bfd9641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f006d200640 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd967d90 0x557bfd9641d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d11e3828e9932ef server_cookie=c6fde9c0f406b165 in_seq=0 out_seq=0 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f004f400640 10 client.14901 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f004f400640 1 -- 172.21.15.7:0/3201131334 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557bfd854260 con 0x557bfd967d90 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f004f400640 10 client.14901 handle_command_reply: tid=0 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f00733ae740 1 client.14901 shutdown 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 msgr2=0x557bfd8f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.381+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0060009f40 tx=0x7f0060000f40 comp rx=0 tx=0).stop 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 shutdown_connections 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557bfd967d90 0x557bfd9641d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557bfd8eb6a0 0x557bfd8f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557bfd8e6530 0x557bfd8f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 --2- 172.21.15.7:0/3201131334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557bfd8e5b40 0x557bfd8f57a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 >> 172.21.15.7:0/3201131334 conn(0x557bfd8e10b0 msgr2=0x557bfd8e1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f004f400640 0 client.14901 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 shutdown_connections 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 1 -- 172.21.15.7:0/3201131334 wait complete. 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f00733ae740 20 client.14901 trim_cache size 0 max 16384 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24559, 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:32:37.385 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 100.45499651199999, 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 122.140862949 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:32:37.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 msgr2=0x7f00500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 0x7f00500a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f005c0ba6c0 tx=0x7f005c0f5940 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 msgr2=0x7f006819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.382+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f005c000c00 tx=0x7f005c005d50 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 shutdown_connections 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f00500a3bb0 0x7f00500a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0068109f90 0x7f00681a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00681095d0 0x7f006819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 --2- 172.21.15.7:0/3283609390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00681083e0 0x7f006819e750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 >> 172.21.15.7:0/3283609390 conn(0x7f00680773a0 msgr2=0x7f006810c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 shutdown_connections 2024-04-20T21:32:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.383+0000 7f006f000640 1 -- 172.21.15.7:0/3283609390 wait complete. 2024-04-20T21:32:37.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1648788039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.395 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4183393109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:32:37.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1648788039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4183393109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1648788039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4183393109' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:32:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.606+0000 7f26d5c00640 1 -- 172.21.15.7:0/163159094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0102850 msgr2=0x7f26d0104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.606+0000 7f26d5c00640 1 --2- 172.21.15.7:0/163159094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0102850 0x7f26d0104c40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009ca0 tx=0x7f26c002c100 comp rx=0 tx=0).stop 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 -- 172.21.15.7:0/163159094 shutdown_connections 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 --2- 172.21.15.7:0/163159094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0107ab0 0x7f26d0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 --2- 172.21.15.7:0/163159094 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0105180 0x7f26d0107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 --2- 172.21.15.7:0/163159094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0102850 0x7f26d0104c40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 -- 172.21.15.7:0/163159094 >> 172.21.15.7:0/163159094 conn(0x7f26d00fc240 msgr2=0x7f26d00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 -- 172.21.15.7:0/163159094 shutdown_connections 2024-04-20T21:32:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.607+0000 7f26d5c00640 1 -- 172.21.15.7:0/163159094 wait complete. 2024-04-20T21:32:37.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.608+0000 7f26d5c00640 1 Processor -- start 2024-04-20T21:32:37.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.608+0000 7f26d5c00640 1 -- start start 2024-04-20T21:32:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 0x7f26d019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 0x7f26d019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0107ab0 0x7f26d01a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010da80 con 0x7f26d0105180 2024-04-20T21:32:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010d900 con 0x7f26d0107ab0 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010dc00 con 0x7f26d0102850 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 0x7f26d019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 0x7f26d019a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53982/0 (socket says 172.21.15.7:53982) 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26d5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0107ab0 0x7f26d01a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cf400640 1 -- 172.21.15.7:0/4005151571 learned_addr learned my addr 172.21.15.7:0/4005151571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 0x7f26d019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cfe00640 1 -- 172.21.15.7:0/4005151571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0107ab0 msgr2=0x7f26d01a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.609+0000 7f26cfe00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0107ab0 0x7f26d01a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26cfe00640 1 -- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 msgr2=0x7f26d019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26cfe00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 0x7f26d019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26cfe00640 1 -- 172.21.15.7:0/4005151571 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d01a1f50 con 0x7f26d0102850 2024-04-20T21:32:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26cfe00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 0x7f26d019a330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009bf0 tx=0x7f26c0035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26ccc00640 1 -- 172.21.15.7:0/4005151571 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0039040 con 0x7f26d0102850 2024-04-20T21:32:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26ccc00640 1 -- 172.21.15.7:0/4005151571 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c0003970 con 0x7f26d0102850 2024-04-20T21:32:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26ccc00640 1 -- 172.21.15.7:0/4005151571 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0004360 con 0x7f26d0102850 2024-04-20T21:32:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d01a21e0 con 0x7f26d0102850 2024-04-20T21:32:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.610+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d01a2650 con 0x7f26d0102850 2024-04-20T21:32:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.613+0000 7f26da026740 1 Processor -- start 2024-04-20T21:32:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 -- start start 2024-04-20T21:32:37.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26ccc00640 1 -- 172.21.15.7:0/4005151571 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f26c0046030 con 0x7f26d0102850 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x56500599e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56500599e6a0 0x5650059a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650058ff9d0 con 0x56500599e6a0 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650058a6270 con 0x565005998b40 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26da026740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650058fb990 con 0x565005999530 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26d5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26d5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38470/0 (socket says 172.21.15.7:38470) 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26d5200640 1 -- 172.21.15.7:0/2352001541 learned_addr learned my addr 172.21.15.7:0/2352001541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26cf400640 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56500599e6a0 0x5650059a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.614+0000 7f26cfe00640 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x56500599e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 -- 172.21.15.7:0/2352001541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 msgr2=0x56500599e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x56500599e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 -- 172.21.15.7:0/2352001541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56500599e6a0 msgr2=0x5650059a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56500599e6a0 0x5650059a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 -- 172.21.15.7:0/2352001541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650056122d0 con 0x565005998b40 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26d5200640 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009ca0 tx=0x7f26c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ef87c615bb5d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26c5600640 1 -- 172.21.15.7:0/2352001541 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c801e070 con 0x565005998b40 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26c5600640 1 -- 172.21.15.7:0/2352001541 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8009e50 con 0x565005998b40 2024-04-20T21:32:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.615+0000 7f26c5600640 1 -- 172.21.15.7:0/2352001541 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c800ea80 con 0x565005998b40 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26ccc00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 0x7f26b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26ccc00640 1 -- 172.21.15.7:0/4005151571 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f26c00f1ed0 con 0x7f26d0102850 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26cfe00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 0x7f26b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- 172.21.15.7:0/2352001541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 msgr2=0x565005998f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009ca0 tx=0x7f26c8011810 comp rx=0 tx=0).stop 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- 172.21.15.7:0/2352001541 shutdown_connections 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56500599e6a0 0x5650059a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x56500599e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 --2- 172.21.15.7:0/2352001541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005998f60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- 172.21.15.7:0/2352001541 >> 172.21.15.7:0/2352001541 conn(0x5650059940b0 msgr2=0x5650059944b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- 172.21.15.7:0/2352001541 shutdown_connections 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- 172.21.15.7:0/2352001541 wait complete. 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26cfe00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 0x7f26b40a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26c00efd70 tx=0x7f26c002c100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.616+0000 7f26da026740 1 -- start start 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 Processor -- start 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x5650059a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500599e6a0 0x5650059aa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650059a1650 con 0x565005998b40 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650058b95e0 con 0x56500599e6a0 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650059a17d0 con 0x565005999530 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26da026740 10 client.0 _ll_register_callbacks cb 0x565005611ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54004/0 (socket says 172.21.15.7:54004) 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.617+0000 7f26d5200640 1 -- 172.21.15.7:0/2276466866 learned_addr learned my addr 172.21.15.7:0/2276466866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26cf400640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x5650059a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500599e6a0 0x5650059aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 -- 172.21.15.7:0/2276466866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 msgr2=0x5650059a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x5650059a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 -- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500599e6a0 msgr2=0x5650059aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500599e6a0 0x5650059aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 -- 172.21.15.7:0/2276466866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650056dc6b0 con 0x565005998b40 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26d5200640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26c8011d10 tx=0x7f26c800e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f3d7d501f22dea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26be000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c801e040 con 0x565005998b40 2024-04-20T21:32:37.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8009e50 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.618+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f26c8004220 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f26c8004420 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 10 client.14913 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5650059ad9a0 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8009e50 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f26c8004d40 con 0x565005998b40 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 10 client.14913 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 10 client.14913 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 20 client.14913 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5650059afce0 0x5650059b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 4 client.14913 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:32:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.619+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56500590d3f0 con 0x5650059afce0 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.620+0000 7f26cf400640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5650059afce0 0x5650059b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.620+0000 7f26cf400640 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5650059afce0 0x5650059b1150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92361ff1f62dd74 server_cookie=b1b1e131e3aa71ea in_seq=0 out_seq=0 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.620+0000 7f26be000640 10 client.14913 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26be000640 1 -- 172.21.15.7:0/2276466866 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56500590d3f0 con 0x5650059afce0 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26be000640 10 client.14913 handle_command_reply: tid=0 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26da026740 1 client.14913 shutdown 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 msgr2=0x5650059a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26c8011d10 tx=0x7f26c800e460 comp rx=0 tx=0).stop 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.621+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 shutdown_connections 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56500599e6a0 0x5650059aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5650059afce0 0x5650059b1150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005999530 0x5650059a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 --2- 172.21.15.7:0/2276466866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005998b40 0x5650059a30b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 >> 172.21.15.7:0/2276466866 conn(0x5650059940b0 msgr2=0x5650059933f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26be000640 0 client.14913 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:32:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 shutdown_connections 2024-04-20T21:32:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 1 -- 172.21.15.7:0/2276466866 wait complete. 2024-04-20T21:32:37.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.622+0000 7f26da026740 20 client.14913 trim_cache size 0 max 16384 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.641+0000 7f26da026740 1 Processor -- start 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.641+0000 7f26da026740 1 -- start start 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650059940b0 0x5650059a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 0x5650059aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650059af490 con 0x565005999530 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650059afb60 con 0x5650059940b0 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26da026740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650059b0230 con 0x565005998b40 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54006/0 (socket says 172.21.15.7:54006) 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 -- 172.21.15.7:0/3041196977 learned_addr learned my addr 172.21.15.7:0/3041196977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650059940b0 0x5650059a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26cf400640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 0x5650059aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 -- 172.21.15.7:0/3041196977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 msgr2=0x5650059aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.642+0000 7f26d5200640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 0x5650059aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26d5200640 1 -- 172.21.15.7:0/3041196977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650059940b0 msgr2=0x5650059a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26d5200640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650059940b0 0x5650059a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26d5200640 1 -- 172.21.15.7:0/3041196977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56500598fb20 con 0x565005999530 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26cf400640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 0x5650059aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26d5200640 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e430 tx=0x7f26c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3855793009e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26c4c00640 1 -- 172.21.15.7:0/3041196977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c801e070 con 0x565005999530 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26c4c00640 1 -- 172.21.15.7:0/3041196977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8039050 con 0x565005999530 2024-04-20T21:32:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.643+0000 7f26c4c00640 1 -- 172.21.15.7:0/3041196977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004c20 con 0x565005999530 2024-04-20T21:32:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- 172.21.15.7:0/3041196977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 msgr2=0x565005994490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e430 tx=0x7f26c8011810 comp rx=0 tx=0).stop 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- 172.21.15.7:0/3041196977 shutdown_connections 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005994490 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x565005998b40 0x5650059aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 --2- 172.21.15.7:0/3041196977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650059940b0 0x5650059a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- 172.21.15.7:0/3041196977 >> 172.21.15.7:0/3041196977 conn(0x5650059af0b0 msgr2=0x5650059a1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- 172.21.15.7:0/3041196977 shutdown_connections 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- 172.21.15.7:0/3041196977 wait complete. 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 -- start start 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.644+0000 7f26da026740 1 Processor -- start 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650059940b0 0x5650059a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005a11c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5650059b0fd0 con 0x565005999530 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650059b0900 con 0x565005998b40 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5650059b16a0 con 0x5650059940b0 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26da026740 10 client.0 _ll_register_callbacks cb 0x7f26c801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26cf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005a11c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54016/0 (socket says 172.21.15.7:54016) 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 -- 172.21.15.7:0/3001817259 learned_addr learned my addr 172.21.15.7:0/3001817259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650059940b0 0x5650059a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 -- 172.21.15.7:0/3001817259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650059940b0 msgr2=0x5650059a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650059940b0 0x5650059a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 -- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 msgr2=0x565005a11c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:32:37.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005a11c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.645+0000 7f26d5200640 1 -- 172.21.15.7:0/3001817259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565005a14860 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26d5200640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26c8006010 tx=0x7f26c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f49b8d42517c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26be000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c801e030 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f26c8003940 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8039050 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f26c8003f30 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26da026740 10 client.14919 fetch_fsmap learned FSMap version 23 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.646+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x565005a15b60 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26c8039050 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f26c803d070 con 0x565005999530 2024-04-20T21:32:37.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 10 client.14919 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 10 client.14919 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 20 client.14919 populate_metadata read hostname 'smithi007' 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56500599e6a0 0x565005a19100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 4 client.14919 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "status"}) -- 0x565005907260 con 0x56500599e6a0 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.647+0000 7f26cf400640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56500599e6a0 0x565005a19100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26cf400640 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56500599e6a0 0x565005a19100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bba35424d16d801 server_cookie=66f693f070fc86e1 in_seq=0 out_seq=0 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26be000640 10 client.14919 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26be000640 1 -- 172.21.15.7:0/3001817259 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x565005907260 con 0x56500599e6a0 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26be000640 10 client.14919 handle_command_reply: tid=0 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 client.14919 shutdown 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 msgr2=0x565005a141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f26c8006010 tx=0x7f26c8009260 comp rx=0 tx=0).stop 2024-04-20T21:32:37.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 shutdown_connections 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565005998b40 0x565005a11c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56500599e6a0 0x565005a19100 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565005999530 0x565005a141c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 --2- 172.21.15.7:0/3001817259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5650059940b0 0x5650059a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26be000640 0 client.14919 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 >> 172.21.15.7:0/3001817259 conn(0x5650059af0b0 msgr2=0x565005994490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.648+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 shutdown_connections 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26da026740 1 -- 172.21.15.7:0/3001817259 wait complete. 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26da026740 20 client.14919 trim_cache size 0 max 16384 2024-04-20T21:32:37.652 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24575, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 103.770431946, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 111.213437145 2024-04-20T21:32:37.653 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 msgr2=0x7f26b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 0x7f26b40a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f26c00efd70 tx=0x7f26c002c100 comp rx=0 tx=0).stop 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 msgr2=0x7f26d019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 0x7f26d019a330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009bf0 tx=0x7f26c0035d10 comp rx=0 tx=0).stop 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 shutdown_connections 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f26b40a3bb0 0x7f26b40a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26d0107ab0 0x7f26d01a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26d0105180 0x7f26d019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 --2- 172.21.15.7:0/4005151571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26d0102850 0x7f26d019a330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:37.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 >> 172.21.15.7:0/4005151571 conn(0x7f26d00fc240 msgr2=0x7f26d0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:37.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 shutdown_connections 2024-04-20T21:32:37.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:37.649+0000 7f26d5c00640 1 -- 172.21.15.7:0/4005151571 wait complete. 2024-04-20T21:32:37.666 INFO:tasks.cephadm:Setup complete, yielding 2024-04-20T21:32:37.666 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T21:32:37.676 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:32:37.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-20T21:32:38.131 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:38 smithi016 ceph-mon[21409]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:38.462 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:38 smithi007 ceph-mon[16867]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:38.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:38 smithi186 ceph-mon[22157]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:40.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:40 smithi007 ceph-mon[16867]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-20T21:32:40.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:40 smithi016 ceph-mon[21409]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-20T21:32:40.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:40 smithi186 ceph-mon[22157]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 -- 172.21.15.7:0/1973678718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540103540 msgr2=0x7f9540105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 --2- 172.21.15.7:0/1973678718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540103540 0x7f9540105930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9530009990 tx=0x7f9530009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 -- 172.21.15.7:0/1973678718 shutdown_connections 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 --2- 172.21.15.7:0/1973678718 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95401087a0 0x7f954010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 --2- 172.21.15.7:0/1973678718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540103540 0x7f9540105930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 --2- 172.21.15.7:0/1973678718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9540105e70 0x7f9540108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 -- 172.21.15.7:0/1973678718 >> 172.21.15.7:0/1973678718 conn(0x7f95400fcf70 msgr2=0x7f95400ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 -- 172.21.15.7:0/1973678718 shutdown_connections 2024-04-20T21:32:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.492+0000 7f9546800700 1 -- 172.21.15.7:0/1973678718 wait complete. 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.493+0000 7f9546800700 1 Processor -- start 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.493+0000 7f9546800700 1 -- start start 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540105e70 0x7f954019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95401087a0 0x7f95401a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f954010e700 con 0x7f9540105e70 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f954010e580 con 0x7f95401087a0 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f9546800700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f954010e880 con 0x7f9540103540 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f953f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f953ea00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540105e70 0x7f954019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f953f400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:43394/0 (socket says 172.21.15.7:43394) 2024-04-20T21:32:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f953f400700 1 -- 172.21.15.7:0/2440707997 learned_addr learned my addr 172.21.15.7:0/2440707997 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.494+0000 7f953fe00700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95401087a0 0x7f95401a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 -- 172.21.15.7:0/2440707997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95401087a0 msgr2=0x7f95401a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95401087a0 0x7f95401a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 -- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540105e70 msgr2=0x7f954019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540105e70 0x7f954019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 -- 172.21.15.7:0/2440707997 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a2b40 con 0x7f9540103540 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f953f400700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9530005910 tx=0x7f953000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530027070 con 0x7f9540103540 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a2dd0 con 0x7f9540103540 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530022470 con 0x7f9540103540 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953000f6c0 con 0x7f9540103540 2024-04-20T21:32:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.495+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a3170 con 0x7f9540103540 2024-04-20T21:32:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.497+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540069740 con 0x7f9540103540 2024-04-20T21:32:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.498+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f953002a070 con 0x7f9540103540 2024-04-20T21:32:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.500+0000 7f9537e00700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 0x7f95240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.500+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f95300d1300 con 0x7f9540103540 2024-04-20T21:32:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.500+0000 7f953ea00700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 0x7f95240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.501+0000 7f953ea00700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 0x7f95240a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f952c009c60 tx=0x7f952c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.504+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f9530013070 con 0x7f9540103540 2024-04-20T21:32:40.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.760+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9540062e90 con 0x7f95240a3c40 2024-04-20T21:32:40.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.762+0000 7f9537e00700 1 -- 172.21.15.7:0/2440707997 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9540062e90 con 0x7f95240a3c40 2024-04-20T21:32:40.761 INFO:teuthology.orchestra.run.smithi007.stdout:Backend: cephadm 2024-04-20T21:32:40.761 INFO:teuthology.orchestra.run.smithi007.stdout:Available: Yes 2024-04-20T21:32:40.762 INFO:teuthology.orchestra.run.smithi007.stdout:Paused: No 2024-04-20T21:32:40.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.765+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 msgr2=0x7f95240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.765+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 0x7f95240a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f952c009c60 tx=0x7f952c009260 comp rx=0 tx=0).stop 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.765+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 msgr2=0x7f954019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.765+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9530005910 tx=0x7f953000bbb0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 shutdown_connections 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95240a3c40 0x7f95240a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9540103540 0x7f954019aee0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95401087a0 0x7f95401a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 --2- 172.21.15.7:0/2440707997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9540105e70 0x7f954019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:40.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 >> 172.21.15.7:0/2440707997 conn(0x7f95400fcf70 msgr2=0x7f9540102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 shutdown_connections 2024-04-20T21:32:40.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:40.766+0000 7f9546800700 1 -- 172.21.15.7:0/2440707997 wait complete. 2024-04-20T21:32:41.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-20T21:32:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:42.419 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:42 smithi007 ceph-mon[16867]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:32:42.419 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:42 smithi007 ceph-mon[16867]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:42 smithi016 ceph-mon[21409]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:32:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:42 smithi016 ceph-mon[21409]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:42 smithi186 ceph-mon[22157]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:32:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:42 smithi186 ceph-mon[22157]: from='client.25042 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.301+0000 7f9499600700 1 -- 172.21.15.7:0/3372491054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 msgr2=0x7f9494074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:44.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.301+0000 7f9499600700 1 --2- 172.21.15.7:0/3372491054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f9494074ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f9484009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 -- 172.21.15.7:0/3372491054 shutdown_connections 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 --2- 172.21.15.7:0/3372491054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f949410f0a0 0x7f9494111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 --2- 172.21.15.7:0/3372491054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f9494074ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 --2- 172.21.15.7:0/3372491054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94940754b0 0x7f9494075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 -- 172.21.15.7:0/3372491054 >> 172.21.15.7:0/3372491054 conn(0x7f94940fe020 msgr2=0x7f94941004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 -- 172.21.15.7:0/3372491054 shutdown_connections 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.302+0000 7f9499600700 1 -- 172.21.15.7:0/3372491054 wait complete. 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 Processor -- start 2024-04-20T21:32:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 -- start start 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f949419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949410f0a0 0x7f94941a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114cd0 con 0x7f94940754b0 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114b50 con 0x7f9494076a80 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.303+0000 7f9499600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9494114e50 con 0x7f949410f0a0 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58652/0 (socket says 172.21.15.7:58652) 2024-04-20T21:32:44.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 -- 172.21.15.7:0/2508905036 learned_addr learned my addr 172.21.15.7:0/2508905036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f948a000700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f949419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9493400700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949410f0a0 0x7f94941a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 -- 172.21.15.7:0/2508905036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949410f0a0 msgr2=0x7f94941a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949410f0a0 0x7f94941a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 -- 172.21.15.7:0/2508905036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 msgr2=0x7f949419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f949419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 -- 172.21.15.7:0/2508905036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94941a6f70 con 0x7f94940754b0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f948a000700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f949419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.304+0000 7f9492a00700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9484005f50 tx=0x7f948400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.305+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484026070 con 0x7f94940754b0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.305+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94941a7200 con 0x7f94940754b0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.305+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948400f460 con 0x7f94940754b0 2024-04-20T21:32:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.305+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484021670 con 0x7f94940754b0 2024-04-20T21:32:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.305+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94941a7650 con 0x7f94940754b0 2024-04-20T21:32:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.306+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9494069740 con 0x7f94940754b0 2024-04-20T21:32:44.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.308+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f9484029070 con 0x7f94940754b0 2024-04-20T21:32:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.310+0000 7f948be00700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 0x7f94780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.310+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f94840d1240 con 0x7f94940754b0 2024-04-20T21:32:44.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.310+0000 7f948a000700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 0x7f94780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:44.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.311+0000 7f948a000700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 0x7f94780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94940767b0 tx=0x7f947c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.314+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f948405dff0 con 0x7f94940754b0 2024-04-20T21:32:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:44 smithi016 ceph-mon[21409]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:44.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:44 smithi186 ceph-mon[22157]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:44.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:44 smithi007 ceph-mon[16867]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:32:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.557+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9494062e90 con 0x7f94780a3c40 2024-04-20T21:32:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.599+0000 7f948be00700 1 -- 172.21.15.7:0/2508905036 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f9494062e90 con 0x7f94780a3c40 2024-04-20T21:32:44.599 INFO:teuthology.orchestra.run.smithi007.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-20T21:32:44.599 INFO:teuthology.orchestra.run.smithi007.stdout:mds.a smithi007 running (2m) 25s ago 2m 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f1869a100f95 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.b smithi016 running (118s) 49s ago 118s 26.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 92d747a36083 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.c smithi186 running (2m) 49s ago 2m 17.4M - 19.0.0-3133-ga8c84f59 76bc6b91b726 4a0ce48fda1b 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.d smithi007 running (2m) 25s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 d1255bd25d6e 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.e smithi016 running (2m) 49s ago 2m 16.7M - 19.0.0-3133-ga8c84f59 76bc6b91b726 f54d62582455 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.f smithi186 running (2m) 49s ago 2m 18.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 fc2064cada42 2024-04-20T21:32:44.600 INFO:teuthology.orchestra.run.smithi007.stdout:mds.g smithi007 running (2m) 25s ago 2m 12.3M - 19.0.0-3133-ga8c84f59 76bc6b91b726 96fe345f286f 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mds.h smithi016 running (2m) 49s ago 2m 17.0M - 19.0.0-3133-ga8c84f59 76bc6b91b726 ed072abb3a76 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mds.i smithi186 running (2m) 49s ago 2m 20.2M - 19.0.0-3133-ga8c84f59 76bc6b91b726 08bc8d6ae2a7 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mds.j smithi007 running (2m) 25s ago 2m 12.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 660811f03816 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mds.k smithi016 running (2m) 49s ago 2m 13.8M - 19.0.0-3133-ga8c84f59 76bc6b91b726 2b7dbfab4e26 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mds.l smithi186 running (2m) 49s ago 2m 15.5M - 19.0.0-3133-ga8c84f59 76bc6b91b726 caa98babf1cd 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.x smithi007 *:9283,8765 running (10m) 25s ago 10m 515M - 19.0.0-3133-ga8c84f59 76bc6b91b726 5f1fd8eb7db4 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.y smithi016 *:8443,8765 running (8m) 49s ago 8m 451M - 19.0.0-3133-ga8c84f59 76bc6b91b726 40b3af6b6ee6 2024-04-20T21:32:44.601 INFO:teuthology.orchestra.run.smithi007.stdout:mgr.z smithi186 *:8443,8765 running (8m) 49s ago 8m 452M - 19.0.0-3133-ga8c84f59 76bc6b91b726 875a94fb182f 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a smithi007 running (10m) 25s ago 11m 49.7M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 42c2fc8e5ad3 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b smithi016 running (8m) 49s ago 8m 40.6M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 1ec276f2aa16 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c smithi186 running (9m) 49s ago 9m 40.4M 2048M 19.0.0-3133-ga8c84f59 76bc6b91b726 49ba9ff3c181 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 smithi007 running (8m) 25s ago 8m 56.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 4500558dd979 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 smithi016 running (7m) 49s ago 7m 44.7M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 a30fadec1458 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 smithi186 running (7m) 49s ago 7m 56.5M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 9e90caf32b49 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 smithi007 running (6m) 25s ago 6m 42.0M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 18af50c68431 2024-04-20T21:32:44.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 smithi016 running (6m) 49s ago 6m 41.6M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 3b257c22305b 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 smithi186 running (5m) 49s ago 5m 43.4M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 970768e2f751 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 smithi007 running (5m) 25s ago 5m 40.8M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 7adaac2fb1e7 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 smithi016 running (4m) 49s ago 4m 42.8M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 b8bc6d42c0ae 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 smithi186 running (4m) 49s ago 4m 45.6M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 97d706d9c4b9 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 smithi007 running (3m) 25s ago 3m 52.5M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 b842bfb6bd14 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 smithi016 running (3m) 49s ago 3m 52.6M 4273M 19.0.0-3133-ga8c84f59 76bc6b91b726 81bf10d33ae4 2024-04-20T21:32:44.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 smithi186 running (2m) 49s ago 2m 40.9M 4272M 19.0.0-3133-ga8c84f59 76bc6b91b726 c6005db1a0ba 2024-04-20T21:32:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 msgr2=0x7f94780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 0x7f94780a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94940767b0 tx=0x7f947c009260 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 msgr2=0x7f949419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9484005f50 tx=0x7f948400bbb0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 shutdown_connections 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f94780a3c40 0x7f94780a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f949410f0a0 0x7f94941a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94940754b0 0x7f949419f310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 --2- 172.21.15.7:0/2508905036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9494076a80 0x7f949419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.603+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 >> 172.21.15.7:0/2508905036 conn(0x7f94940fe020 msgr2=0x7f9494102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.604+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 shutdown_connections 2024-04-20T21:32:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:44.604+0000 7f9499600700 1 -- 172.21.15.7:0/2508905036 wait complete. 2024-04-20T21:32:45.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-20T21:32:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:46 smithi016 ceph-mon[21409]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:46 smithi016 ceph-mon[21409]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:46.445 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:46 smithi007 ceph-mon[16867]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:46.445 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:46 smithi007 ceph-mon[16867]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:46 smithi186 ceph-mon[22157]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:46.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:46 smithi186 ceph-mon[22157]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.153+0000 7fac23600700 1 -- 172.21.15.7:0/3068580824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 msgr2=0x7fac1c109510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.153+0000 7fac23600700 1 --2- 172.21.15.7:0/3068580824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c109510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fac100098f0 tx=0x7fac10009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:48.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 -- 172.21.15.7:0/3068580824 shutdown_connections 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 --2- 172.21.15.7:0/3068580824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c109510 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 --2- 172.21.15.7:0/3068580824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10ac90 0x7fac1c111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 --2- 172.21.15.7:0/3068580824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10a2f0 0x7fac1c10a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 -- 172.21.15.7:0/3068580824 >> 172.21.15.7:0/3068580824 conn(0x7fac1c0780c0 msgr2=0x7fac1c07a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 -- 172.21.15.7:0/3068580824 shutdown_connections 2024-04-20T21:32:48.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.154+0000 7fac23600700 1 -- 172.21.15.7:0/3068580824 wait complete. 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 Processor -- start 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 -- start start 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 0x7fac1c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10ac90 0x7fac1c1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c114d20 con 0x7fac1c10a2f0 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c114ba0 con 0x7fac1c10ac90 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.155+0000 7fac23600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c114ea0 con 0x7fac1c1090f0 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac1be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 0x7fac1c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41862/0 (socket says 172.21.15.7:41862) 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 -- 172.21.15.7:0/1844472177 learned_addr learned my addr 172.21.15.7:0/1844472177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac1be00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 0x7fac1c19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58678/0 (socket says 172.21.15.7:58678) 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac21600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10ac90 0x7fac1c1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 -- 172.21.15.7:0/1844472177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10ac90 msgr2=0x7fac1c1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10ac90 0x7fac1c1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 -- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 msgr2=0x7fac1c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 0x7fac1c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.156+0000 7fac20c00700 1 -- 172.21.15.7:0/1844472177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1a6fa0 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac20c00700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1000f950 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1000ba40 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10017400 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1a7230 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.157+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a7680 con 0x7fac1c1090f0 2024-04-20T21:32:48.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.158+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c069740 con 0x7fac1c1090f0 2024-04-20T21:32:48.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.160+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fac10003680 con 0x7fac1c1090f0 2024-04-20T21:32:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.162+0000 7fac19600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 0x7fac040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.162+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fac100d1330 con 0x7fac1c1090f0 2024-04-20T21:32:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.162+0000 7fac1be00700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 0x7fac040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:48.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.163+0000 7fac1be00700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 0x7fac040a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac08005fd0 tx=0x7fac08009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:48.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.166+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fac10013030 con 0x7fac1c1090f0 2024-04-20T21:32:48.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:48 smithi007 ceph-mon[16867]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:48.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.408+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fac1c062e90 con 0x7fac040a3c40 2024-04-20T21:32:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.419+0000 7fac19600700 1 -- 172.21.15.7:0/1844472177 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fac1c062e90 con 0x7fac040a3c40 2024-04-20T21:32:48.418 INFO:teuthology.orchestra.run.smithi007.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-20T21:32:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:mds.all 12/12 53s ago 2m smithi007=a;smithi007=d;smithi007=g;smithi007=j;smithi016=b;smithi016=e;smithi016=h;smithi016=k;smithi186=c;smithi186=f;smithi186=i;smithi186=l;count:12 2024-04-20T21:32:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:mgr 3/3 53s ago 8m smithi007=x;smithi016=y;smithi186=z;count:3 2024-04-20T21:32:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:mon 3/3 53s ago 9m smithi007:172.21.15.7=a;smithi016:172.21.15.16=b;smithi186:172.21.15.186=c;count:3 2024-04-20T21:32:48.419 INFO:teuthology.orchestra.run.smithi007.stdout:osd 12 53s ago - 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.422+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 msgr2=0x7fac040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.422+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 0x7fac040a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac08005fd0 tx=0x7fac08009310 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.422+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 msgr2=0x7fac1c19f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.422+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100052f0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 shutdown_connections 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fac040a3c40 0x7fac040a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac1c1090f0 0x7fac1c19f340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac1c10ac90 0x7fac1c1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 --2- 172.21.15.7:0/1844472177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c10a2f0 0x7fac1c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 >> 172.21.15.7:0/1844472177 conn(0x7fac1c0780c0 msgr2=0x7fac1c10d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 shutdown_connections 2024-04-20T21:32:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:48.423+0000 7fac23600700 1 -- 172.21.15.7:0/1844472177 wait complete. 2024-04-20T21:32:48.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:48 smithi016 ceph-mon[21409]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:48.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:48 smithi186 ceph-mon[22157]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:49.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-20T21:32:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:50.341 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:50 smithi007 ceph-mon[16867]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:32:50.341 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:50 smithi007 ceph-mon[16867]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:50.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:50 smithi016 ceph-mon[21409]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:32:50.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:50 smithi016 ceph-mon[21409]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:50 smithi186 ceph-mon[22157]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:32:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:50 smithi186 ceph-mon[22157]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.944+0000 7f1a1e400700 1 -- 172.21.15.7:0/1021506748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a18105da0 msgr2=0x7f1a18108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.944+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1021506748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a18105da0 0x7f1a18108190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1a040098f0 tx=0x7f1a04009c00 comp rx=0 tx=0).stop 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 -- 172.21.15.7:0/1021506748 shutdown_connections 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1021506748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a181086d0 0x7f1a1810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1021506748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18103470 0x7f1a18105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1021506748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a18105da0 0x7f1a18108190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 -- 172.21.15.7:0/1021506748 >> 172.21.15.7:0/1021506748 conn(0x7f1a180fce60 msgr2=0x7f1a180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 -- 172.21.15.7:0/1021506748 shutdown_connections 2024-04-20T21:32:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.945+0000 7f1a1e400700 1 -- 172.21.15.7:0/1021506748 wait complete. 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.946+0000 7f1a1e400700 1 Processor -- start 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.946+0000 7f1a1e400700 1 -- start start 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18103470 0x7f1a1819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 0x7f1a181a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810e610 con 0x7f1a18105da0 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810e490 con 0x7f1a18103470 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1e400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810e790 con 0x7f1a181086d0 2024-04-20T21:32:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a17e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a17e00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58708/0 (socket says 172.21.15.7:58708) 2024-04-20T21:32:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a17e00700 1 -- 172.21.15.7:0/1722661008 learned_addr learned my addr 172.21.15.7:0/1722661008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1d000700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18103470 0x7f1a1819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.947+0000 7f1a1da00700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 0x7f1a181a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 -- 172.21.15.7:0/1722661008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 msgr2=0x7f1a181a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 0x7f1a181a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 -- 172.21.15.7:0/1722661008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18103470 msgr2=0x7f1a1819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18103470 0x7f1a1819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 -- 172.21.15.7:0/1722661008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a181a2a50 con 0x7f1a18105da0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a1da00700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 0x7f1a181a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a17e00700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400fca0 tx=0x7f1a04005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04025070 con 0x7f1a18105da0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a181a2ce0 con 0x7f1a18105da0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.948+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a04005650 con 0x7f1a18105da0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.949+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04003d20 con 0x7f1a18105da0 2024-04-20T21:32:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.949+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a3190 con 0x7f1a18105da0 2024-04-20T21:32:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.950+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a18069740 con 0x7f1a18105da0 2024-04-20T21:32:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.952+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1a04028070 con 0x7f1a18105da0 2024-04-20T21:32:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.953+0000 7f1a15600700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 0x7f1a000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.954+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1a040d0e00 con 0x7f1a18105da0 2024-04-20T21:32:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.954+0000 7f1a1d000700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 0x7f1a000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.955+0000 7f1a1d000700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 0x7f1a000a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005fd0 tx=0x7f1a0c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:51.958+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1a0405cff0 con 0x7f1a18105da0 2024-04-20T21:32:52.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.198+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1a18062e90 con 0x7f1a000a3c40 2024-04-20T21:32:52.197 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:52 smithi007 ceph-mon[16867]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.200+0000 7f1a15600700 1 -- 172.21.15.7:0/1722661008 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1a18062e90 con 0x7f1a000a3c40 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stdout:HOST ADDR LABELS STATUS 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 172.21.15.7 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 172.21.15.16 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stdout:smithi186 172.21.15.186 2024-04-20T21:32:52.199 INFO:teuthology.orchestra.run.smithi007.stdout:3 hosts in cluster 2024-04-20T21:32:52.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 msgr2=0x7f1a000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:52.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 0x7f1a000a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005fd0 tx=0x7f1a0c009310 comp rx=0 tx=0).stop 2024-04-20T21:32:52.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 msgr2=0x7f1a1819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400fca0 tx=0x7f1a04005250 comp rx=0 tx=0).stop 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 shutdown_connections 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1a000a3c40 0x7f1a000a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1a181086d0 0x7f1a181a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.203+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18103470 0x7f1a1819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.204+0000 7f1a1e400700 1 --2- 172.21.15.7:0/1722661008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a18105da0 0x7f1a1819b330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.204+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 >> 172.21.15.7:0/1722661008 conn(0x7f1a180fce60 msgr2=0x7f1a18102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.204+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 shutdown_connections 2024-04-20T21:32:52.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:52.204+0000 7f1a1e400700 1 -- 172.21.15.7:0/1722661008 wait complete. 2024-04-20T21:32:52.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:52 smithi016 ceph-mon[21409]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:52.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:52 smithi186 ceph-mon[22157]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:32:53.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-20T21:32:53.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:53 smithi016 ceph-mon[21409]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:53.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:53 smithi186 ceph-mon[22157]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:53.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:53 smithi007 ceph-mon[16867]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:53.629 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:54.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:54 smithi016 ceph-mon[21409]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:54 smithi186 ceph-mon[22157]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:54.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:54 smithi007 ceph-mon[16867]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 -- 172.21.15.7:0/373792120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 msgr2=0x7fe72c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 --2- 172.21.15.7:0/373792120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c101750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009990 tx=0x7fe71c009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 -- 172.21.15.7:0/373792120 shutdown_connections 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 --2- 172.21.15.7:0/373792120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 --2- 172.21.15.7:0/373792120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c101750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 --2- 172.21.15.7:0/373792120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 -- 172.21.15.7:0/373792120 >> 172.21.15.7:0/373792120 conn(0x7fe72c0fcf50 msgr2=0x7fe72c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 -- 172.21.15.7:0/373792120 shutdown_connections 2024-04-20T21:32:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.996+0000 7fe731600700 1 -- 172.21.15.7:0/373792120 wait complete. 2024-04-20T21:32:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.997+0000 7fe731600700 1 Processor -- start 2024-04-20T21:32:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 -- start start 2024-04-20T21:32:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c1a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c110850 con 0x7fe72c102650 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1106d0 con 0x7fe72c101330 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe731600700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1109d0 con 0x7fe72c101c90 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe72b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c1a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.998+0000 7fe72b400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c1a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54870/0 (socket says 172.21.15.7:54870) 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe72b400700 1 -- 172.21.15.7:0/4149979283 learned_addr learned my addr 172.21.15.7:0/4149979283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe72aa00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 -- 172.21.15.7:0/4149979283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 msgr2=0x7fe72c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 -- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 msgr2=0x7fe72c1a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c1a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 -- 172.21.15.7:0/4149979283 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c1a2b60 con 0x7fe72c101c90 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:55.999+0000 7fe723e00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c19b440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ea20 tx=0x7fe71400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.000+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714004220 con 0x7fe72c101c90 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.000+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71400ce50 con 0x7fe72c101c90 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.000+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c1a2e50 con 0x7fe72c101c90 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.000+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714003dc0 con 0x7fe72c101c90 2024-04-20T21:32:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.000+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c1a3350 con 0x7fe72c101c90 2024-04-20T21:32:56.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.001+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c069740 con 0x7fe72c101c90 2024-04-20T21:32:56.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.004+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fe71400c970 con 0x7fe72c101c90 2024-04-20T21:32:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.005+0000 7fe723400700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 0x7fe7100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.005+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe7140d06b0 con 0x7fe72c101c90 2024-04-20T21:32:56.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.006+0000 7fe72aa00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 0x7fe7100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.006+0000 7fe72aa00700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 0x7fe7100a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b7b0 tx=0x7fe71c00b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:56.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.007+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe71409d060 con 0x7fe72c101c90 2024-04-20T21:32:56.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.246+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 --> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe72c062e90 con 0x7fe7100a3cc0 2024-04-20T21:32:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.250+0000 7fe723400700 1 -- 172.21.15.7:0/4149979283 <== mgr.14152 v2:172.21.15.7:6800/2375060083 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe72c062e90 con 0x7fe7100a3cc0 2024-04-20T21:32:56.250 INFO:teuthology.orchestra.run.smithi007.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-20T21:32:56.250 INFO:teuthology.orchestra.run.smithi007.stdout:smithi007 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GE480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-20T21:32:56.250 INFO:teuthology.orchestra.run.smithi007.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-20T21:32:56.250 INFO:teuthology.orchestra.run.smithi007.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-20T21:32:56.250 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:56 smithi007 ceph-mon[16867]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.253+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 msgr2=0x7fe7100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.253+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 0x7fe7100a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe71c00b7b0 tx=0x7fe71c00b880 comp rx=0 tx=0).stop 2024-04-20T21:32:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.253+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 msgr2=0x7fe72c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.253+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c19b440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ea20 tx=0x7fe71400ed30 comp rx=0 tx=0).stop 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 shutdown_connections 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe72c102650 0x7fe72c1a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe72c101330 0x7fe72c19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe7100a3cc0 0x7fe7100a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 --2- 172.21.15.7:0/4149979283 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe72c101c90 0x7fe72c19b440 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 >> 172.21.15.7:0/4149979283 conn(0x7fe72c0fcf50 msgr2=0x7fe72c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 shutdown_connections 2024-04-20T21:32:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:56.254+0000 7fe731600700 1 -- 172.21.15.7:0/4149979283 wait complete. 2024-04-20T21:32:56.446 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:56 smithi016 ceph-mon[21409]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:56.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:56 smithi186 ceph-mon[22157]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:32:57.016 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-20T21:32:57.027 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:32:57.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-20T21:32:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:32:58.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:58 smithi007 ceph-mon[16867]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:58.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:32:58 smithi007 ceph-mon[16867]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:58.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:58 smithi016 ceph-mon[21409]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:58.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:32:58 smithi016 ceph-mon[21409]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:58 smithi186 ceph-mon[22157]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:32:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:32:58 smithi186 ceph-mon[22157]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 -- 172.21.15.7:0/1803205761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 msgr2=0x7f960c102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 --2- 172.21.15.7:0/1803205761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c102ed0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009990 tx=0x7f95fc009ca0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 -- 172.21.15.7:0/1803205761 shutdown_connections 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 --2- 172.21.15.7:0/1803205761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c102ed0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 --2- 172.21.15.7:0/1803205761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c104650 0x7f960c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 --2- 172.21.15.7:0/1803205761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c103cb0 0x7f960c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 -- 172.21.15.7:0/1803205761 >> 172.21.15.7:0/1803205761 conn(0x7f960c0fe050 msgr2=0x7f960c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 -- 172.21.15.7:0/1803205761 shutdown_connections 2024-04-20T21:32:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.687+0000 7f9610c00700 1 -- 172.21.15.7:0/1803205761 wait complete. 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 Processor -- start 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 -- start start 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c104650 0x7f960c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10e6c0 con 0x7f960c103cb0 2024-04-20T21:32:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10e540 con 0x7f960c104650 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.688+0000 7f9610c00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10e840 con 0x7f960c102ab0 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54882/0 (socket says 172.21.15.7:54882) 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 -- 172.21.15.7:0/432666194 learned_addr learned my addr 172.21.15.7:0/432666194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9609e00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f960a800700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c104650 0x7f960c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 -- 172.21.15.7:0/432666194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 msgr2=0x7f960c198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 -- 172.21.15.7:0/432666194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c104650 msgr2=0x7f960c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c104650 0x7f960c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.689+0000 7f9601400700 1 -- 172.21.15.7:0/432666194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c1a09e0 con 0x7f960c103cb0 2024-04-20T21:32:59.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9601400700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e9a0 tx=0x7f95f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:59.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400ca10 con 0x7f960c103cb0 2024-04-20T21:32:59.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c074ce0 con 0x7f960c103cb0 2024-04-20T21:32:59.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f400cbb0 con 0x7f960c103cb0 2024-04-20T21:32:59.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4018680 con 0x7f960c103cb0 2024-04-20T21:32:59.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.690+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c075270 con 0x7f960c103cb0 2024-04-20T21:32:59.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.692+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c069740 con 0x7f960c103cb0 2024-04-20T21:32:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.694+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f95f4010430 con 0x7f960c103cb0 2024-04-20T21:32:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.695+0000 7f9603400700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 0x7f95ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:32:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.695+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f95f40d05b0 con 0x7f960c103cb0 2024-04-20T21:32:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.695+0000 7f9609e00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 0x7f95ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:32:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.696+0000 7f9609e00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 0x7f95ec0a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f960c19a270 tx=0x7f95fc01e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:32:59.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.700+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f95f409cf60 con 0x7f960c103cb0 2024-04-20T21:32:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.979+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f960c076650 con 0x7f960c103cb0 2024-04-20T21:32:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.980+0000 7f9603400700 1 -- 172.21.15.7:0/432666194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3049 (secure 0 0 0) 0x7f95f409c860 con 0x7f960c103cb0 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:e23 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:legacy client fscid: 1 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:Filesystem 'cephfs' (1) 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:fs_name cephfs 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 23 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-20T21:32:59.980 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-04-20T21:30:52.886616+0000 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-04-20T21:30:58.437063+0000 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:tableserver 0 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:root 0 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:session_timeout 300 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:session_autoclose 300 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:max_file_size 1099511627776 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:max_xattr_size 65536 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:required_client_features {} 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure 0 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:last_failure_osd_epoch 0 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:snap_rstat 1 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:max_mds 3 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:in 0,1,2 2024-04-20T21:32:59.981 INFO:teuthology.orchestra.run.smithi007.stdout:up {0=24575,1=24559,2=24535} 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:failed 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:damaged 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:stopped 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:data_pools [3] 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:metadata_pool 2 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:inline_data disabled 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:balancer 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:bal_rank_mask -1 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:standby_count_wanted 0 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:qdb_cluster leader: 24575 members: 24535,24559,24575 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b{0:24575} state up:active seq 3 addr [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f{0:24523} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/2162825015,v1:172.21.15.186:6835/2162825015] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.982 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.i{1:24559} state up:active seq 7 addr [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.h{1:24521} state up:standby-replay seq 1 addr [v2:172.21.15.16:6836/4051126294,v1:172.21.15.16:6837/4051126294] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c{2:24535} state up:active seq 9 addr [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e{2:24491} state up:standby-replay seq 1 addr [v2:172.21.15.16:6834/2330254135,v1:172.21.15.16:6835/2330254135] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:Standby daemons: 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.g{-1:14448} state up:standby seq 1 addr [v2:172.21.15.7:6838/766857391,v1:172.21.15.7:6839/766857391] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a{-1:14463} state up:standby seq 1 addr [v2:172.21.15.7:6840/771888475,v1:172.21.15.7:6841/771888475] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.16:6832/1030631519,v1:172.21.15.16:6833/1030631519] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.l{-1:24457} state up:standby seq 1 addr [v2:172.21.15.186:6832/3154608825,v1:172.21.15.186:6833/3154608825] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.j{-1:24461} state up:standby seq 1 addr [v2:172.21.15.7:6834/3500904086,v1:172.21.15.7:6835/3500904086] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.983 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d{-1:24476} state up:standby seq 1 addr [v2:172.21.15.7:6836/2021614032,v1:172.21.15.7:6837/2021614032] compat {c=[1],r=[1],i=[1fff]}] 2024-04-20T21:32:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 msgr2=0x7f95ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 0x7f95ec0a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f960c19a270 tx=0x7f95fc01e040 comp rx=0 tx=0).stop 2024-04-20T21:32:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 msgr2=0x7f960c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:32:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95f400e9a0 tx=0x7f95f400ecb0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 shutdown_connections 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f95ec0a3cc0 0x7f95ec0a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f960c102ab0 0x7f960c198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c104650 0x7f960c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.983+0000 7f9610c00700 1 --2- 172.21.15.7:0/432666194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f960c103cb0 0x7f960c1992d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.984+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 >> 172.21.15.7:0/432666194 conn(0x7f960c0fe050 msgr2=0x7f960c1070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.984+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 shutdown_connections 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:32:59.984+0000 7f9610c00700 1 -- 172.21.15.7:0/432666194 wait complete. 2024-04-20T21:32:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:33:00.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:00 smithi007 ceph-mon[16867]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:00.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:00 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/432666194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:00 smithi016 ceph-mon[21409]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:00 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/432666194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:00.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:00 smithi186 ceph-mon[22157]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:00.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:00 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/432666194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-20T21:33:00.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-20T21:33:01.309 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:02.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:02 smithi016 ceph-mon[21409]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:02.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:02 smithi186 ceph-mon[22157]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:02.500 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:02 smithi007 ceph-mon[16867]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 -- 172.21.15.7:0/2730002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 msgr2=0x7f3e34078ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 --2- 172.21.15.7:0/2730002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e34078ad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009990 tx=0x7f3e20009ca0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 -- 172.21.15.7:0/2730002886 shutdown_connections 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 --2- 172.21.15.7:0/2730002886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34079010 0x7f3e34079890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 --2- 172.21.15.7:0/2730002886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34077400 0x7f3e34077820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 --2- 172.21.15.7:0/2730002886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e34078ad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 -- 172.21.15.7:0/2730002886 >> 172.21.15.7:0/2730002886 conn(0x7f3e341002a0 msgr2=0x7f3e341026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 -- 172.21.15.7:0/2730002886 shutdown_connections 2024-04-20T21:33:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.543+0000 7f3e3c400700 1 -- 172.21.15.7:0/2730002886 wait complete. 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.544+0000 7f3e3c400700 1 Processor -- start 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 -- start start 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 0x7f3e3419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34079010 0x7f3e341a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e34114ce0 con 0x7f3e34078650 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e34114b60 con 0x7f3e34079010 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3c400700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e34114e60 con 0x7f3e34077400 2024-04-20T21:33:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e39a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 0x7f3e3419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e39000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e39a00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 0x7f3e3419f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34218/0 (socket says 172.21.15.7:34218) 2024-04-20T21:33:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e39a00700 1 -- 172.21.15.7:0/3231674526 learned_addr learned my addr 172.21.15.7:0/3231674526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e39000700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41064/0 (socket says 172.21.15.7:41064) 2024-04-20T21:33:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.545+0000 7f3e3a400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34079010 0x7f3e341a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 -- 172.21.15.7:0/3231674526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 msgr2=0x7f3e3419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 0x7f3e3419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 -- 172.21.15.7:0/3231674526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34079010 msgr2=0x7f3e341a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34079010 0x7f3e341a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 -- 172.21.15.7:0/3231674526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e341a6f00 con 0x7f3e34078650 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.546+0000 7f3e39000700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e20000c00 tx=0x7f3e2000bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.547+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20026070 con 0x7f3e34078650 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.547+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e20003d70 con 0x7f3e34078650 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.547+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e341a7190 con 0x7f3e34078650 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.547+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20021410 con 0x7f3e34078650 2024-04-20T21:33:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.547+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e341a75e0 con 0x7f3e34078650 2024-04-20T21:33:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.548+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e34069740 con 0x7f3e34078650 2024-04-20T21:33:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.550+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f3e20029070 con 0x7f3e34078650 2024-04-20T21:33:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.552+0000 7f3e32000700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 0x7f3e1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.552+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3e200d1770 con 0x7f3e34078650 2024-04-20T21:33:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.552+0000 7f3e39a00700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 0x7f3e1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:03.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.553+0000 7f3e39a00700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 0x7f3e1c0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009680 tx=0x7f3e28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:03.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.555+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f3e20030080 con 0x7f3e34078650 2024-04-20T21:33:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.787+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3e3402df90 con 0x7f3e34078650 2024-04-20T21:33:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.788+0000 7f3e32000700 1 -- 172.21.15.7:0/3231674526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5212 (secure 0 0 0) 0x7f3e20013070 con 0x7f3e34078650 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 88 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-04-20T21:21:41.203021+0000 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-04-20T21:31:50.134558+0000 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 26 2024-04-20T21:33:03.789 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client luminous 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 12 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6802/2111694387,v1:172.21.15.7:6803/2111694387] [v2:172.21.15.7:6804/2111694387,v1:172.21.15.7:6805/2111694387] exists,up 9f182508-4c5f-4296-9117-738a5d55d302 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6800/3476654955,v1:172.21.15.16:6801/3476654955] [v2:172.21.15.16:6802/3476654955,v1:172.21.15.16:6803/3476654955] exists,up dbdf4c8d-79a8-4077-9214-86fc2a0d993d 2024-04-20T21:33:03.790 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/1038897606,v1:172.21.15.186:6801/1038897606] [v2:172.21.15.186:6802/1038897606,v1:172.21.15.186:6803/1038897606] exists,up 40aadd77-3cf6-4a09-b30a-b038345607eb 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6810/55132163,v1:172.21.15.7:6811/55132163] [v2:172.21.15.7:6812/55132163,v1:172.21.15.7:6813/55132163] exists,up 3058d99a-73d2-459c-ac66-127d373f59fc 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6808/215460715,v1:172.21.15.16:6809/215460715] [v2:172.21.15.16:6810/215460715,v1:172.21.15.16:6811/215460715] exists,up aeed2b10-a459-46cc-b744-38c3c4fb5eb2 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/955879969,v1:172.21.15.186:6809/955879969] [v2:172.21.15.186:6810/955879969,v1:172.21.15.186:6811/955879969] exists,up 19d66739-ba9a-49b9-a99c-7792746f048d 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6818/2723605602,v1:172.21.15.7:6819/2723605602] [v2:172.21.15.7:6820/2723605602,v1:172.21.15.7:6821/2723605602] exists,up 3b410e07-62a3-487c-b3fb-0e71d0ed25a4 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6816/4148897395,v1:172.21.15.16:6817/4148897395] [v2:172.21.15.16:6818/4148897395,v1:172.21.15.16:6819/4148897395] exists,up af75979c-c869-4b74-a32f-975d039d77ff 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/3369879966,v1:172.21.15.186:6817/3369879966] [v2:172.21.15.186:6818/3369879966,v1:172.21.15.186:6819/3369879966] exists,up d25aec8d-214f-44ed-8166-4b0dfde293d5 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6826/3395112642,v1:172.21.15.7:6827/3395112642] [v2:172.21.15.7:6828/3395112642,v1:172.21.15.7:6829/3395112642] exists,up f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6824/2592476634,v1:172.21.15.16:6825/2592476634] [v2:172.21.15.16:6826/2592476634,v1:172.21.15.16:6827/2592476634] exists,up 98497028-afaa-4bae-a644-e31beddd7acd 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/2885587987,v1:172.21.15.186:6825/2885587987] [v2:172.21.15.186:6826/2885587987,v1:172.21.15.186:6827/2885587987] exists,up 2d80cce8-3e29-4afe-b398-5388ebfe1b5c 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.6 [8,11] 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.e [2,1] 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.10 [2,4] 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 2.1c [8,11] 2024-04-20T21:33:03.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.1 [4,9] 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.c [3,9] 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:pg_upmap_items 3.2c [4,9] 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/2119469525 expires 2024-04-21T21:22:48.211353+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/2119469525 expires 2024-04-21T21:22:48.211353+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1792814241 expires 2024-04-21T21:22:48.211353+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1236425601 expires 2024-04-21T21:22:48.211353+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6801/2055239987 expires 2024-04-21T21:22:14.963843+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1608503169 expires 2024-04-21T21:22:14.963843+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/1751658409 expires 2024-04-21T21:22:14.963843+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2563830614 expires 2024-04-21T21:22:48.211353+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:6800/2055239987 expires 2024-04-21T21:22:14.963843+0000 2024-04-20T21:33:03.792 INFO:teuthology.orchestra.run.smithi007.stdout:blocklist 172.21.15.7:0/2403355260 expires 2024-04-21T21:22:14.963843+0000 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.791+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 msgr2=0x7f3e1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 0x7f3e1c0a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009680 tx=0x7f3e28009260 comp rx=0 tx=0).stop 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 msgr2=0x7f3e3419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e20000c00 tx=0x7f3e2000bb90 comp rx=0 tx=0).stop 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 shutdown_connections 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f3e1c0a3c40 0x7f3e1c0a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34077400 0x7f3e3419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e34079010 0x7f3e341a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 --2- 172.21.15.7:0/3231674526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e34078650 0x7f3e3419f7e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 >> 172.21.15.7:0/3231674526 conn(0x7f3e341002a0 msgr2=0x7f3e34101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 shutdown_connections 2024-04-20T21:33:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:03.792+0000 7f3e3c400700 1 -- 172.21.15.7:0/3231674526 wait complete. 2024-04-20T21:33:04.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:04 smithi016 ceph-mon[21409]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:04.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:04 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3231674526' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:04.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:04 smithi186 ceph-mon[22157]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:04.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:04 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3231674526' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:04 smithi007 ceph-mon[16867]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:04.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:04 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3231674526' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-20T21:33:04.639 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-20T21:33:04.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.875+0000 7fd4fcc00640 1 -- 172.21.15.7:0/1558837366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8078500 msgr2=0x7fd4f8078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.875+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/1558837366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8078500 0x7fd4f8078da0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009ca0 tx=0x7fd4f002c100 comp rx=0 tx=0).stop 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 -- 172.21.15.7:0/1558837366 shutdown_connections 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/1558837366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8078500 0x7fd4f8078da0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/1558837366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8077b60 0x7fd4f8077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/1558837366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8076920 0x7fd4f8076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 -- 172.21.15.7:0/1558837366 >> 172.21.15.7:0/1558837366 conn(0x7fd4f80ff7b0 msgr2=0x7fd4f8101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 -- 172.21.15.7:0/1558837366 shutdown_connections 2024-04-20T21:33:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.876+0000 7fd4fcc00640 1 -- 172.21.15.7:0/1558837366 wait complete. 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 Processor -- start 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 -- start start 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 0x7fd4f819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 0x7fd4f819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f8114240 con 0x7fd4f8077b60 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81140c0 con 0x7fd4f8078500 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.877+0000 7fd4fcc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f81143c0 con 0x7fd4f8076920 2024-04-20T21:33:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 0x7fd4f819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 0x7fd4f819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54240/0 (socket says 172.21.15.7:54240) 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 -- 172.21.15.7:0/2927487831 learned_addr learned my addr 172.21.15.7:0/2927487831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 -- 172.21.15.7:0/2927487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 msgr2=0x7fd4f819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 0x7fd4f819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 -- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 msgr2=0x7fd4f819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 0x7fd4f819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f6800640 1 -- 172.21.15.7:0/2927487831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f81a6460 con 0x7fd4f8078500 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4f5e00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 0x7fd4f819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.878+0000 7fd4ed400640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 0x7fd4f819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4f6800640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002c640 tx=0x7fd4f0035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0039040 con 0x7fd4f8078500 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4f0044070 con 0x7fd4f8078500 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f00043c0 con 0x7fd4f8078500 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f81a66f0 con 0x7fd4f8078500 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.879+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f81a6be0 con 0x7fd4f8078500 2024-04-20T21:33:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.880+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f8110b90 con 0x7fd4f8078500 2024-04-20T21:33:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.882+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd4f0004560 con 0x7fd4f8078500 2024-04-20T21:33:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.884+0000 7fd4ef400640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 0x7fd4dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.884+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd4f00f29e0 con 0x7fd4f8078500 2024-04-20T21:33:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.884+0000 7fd4f5e00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 0x7fd4dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.885+0000 7fd4f5e00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 0x7fd4dc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009fc0 tx=0x7fd4e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:04.888+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd4f00bf390 con 0x7fd4f8078500 2024-04-20T21:33:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.097+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4f8062b60 con 0x7fd4f8078500 2024-04-20T21:33:05.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.100+0000 7fd4ef400640 1 -- 172.21.15.7:0/2927487831 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7fd4f00098b0 con 0x7fd4f8078500 2024-04-20T21:33:05.099 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:05.100 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 msgr2=0x7fd4dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 0x7fd4dc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009fc0 tx=0x7fd4e8005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 msgr2=0x7fd4f81a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4f002c640 tx=0x7fd4f0035d10 comp rx=0 tx=0).stop 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 shutdown_connections 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd4dc0a3bb0 0x7fd4dc0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f8078500 0x7fd4f81a5df0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f8077b60 0x7fd4f819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 --2- 172.21.15.7:0/2927487831 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4f8076920 0x7fd4f819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.102+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 >> 172.21.15.7:0/2927487831 conn(0x7fd4f80ff7b0 msgr2=0x7fd4f81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.103+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 shutdown_connections 2024-04-20T21:33:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.103+0000 7fd4fcc00640 1 -- 172.21.15.7:0/2927487831 wait complete. 2024-04-20T21:33:05.123 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:05.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.346+0000 7f2832800640 1 -- 172.21.15.7:0/1107337554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c075f40 msgr2=0x7f282c074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.346+0000 7f2832800640 1 --2- 172.21.15.7:0/1107337554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c075f40 0x7f282c074390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f281c009900 tx=0x7f281c02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 -- 172.21.15.7:0/1107337554 shutdown_connections 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 --2- 172.21.15.7:0/1107337554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 0x7f282c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 --2- 172.21.15.7:0/1107337554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c109e10 0x7f282c10c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 --2- 172.21.15.7:0/1107337554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c075f40 0x7f282c074390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 -- 172.21.15.7:0/1107337554 >> 172.21.15.7:0/1107337554 conn(0x7f282c0fd4b0 msgr2=0x7f282c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 -- 172.21.15.7:0/1107337554 shutdown_connections 2024-04-20T21:33:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.347+0000 7f2832800640 1 -- 172.21.15.7:0/1107337554 wait complete. 2024-04-20T21:33:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 Processor -- start 2024-04-20T21:33:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 -- start start 2024-04-20T21:33:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 0x7f282c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c075f40 0x7f282c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10fe60 con 0x7f282c109e10 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10fce0 con 0x7f282c074960 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f2832800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f282c10ffe0 con 0x7f282c075f40 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f282be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f282be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41100/0 (socket says 172.21.15.7:41100) 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.348+0000 7f282be00640 1 -- 172.21.15.7:0/1530676492 learned_addr learned my addr 172.21.15.7:0/1530676492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282b400640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 0x7f282c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282aa00640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c075f40 0x7f282c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 -- 172.21.15.7:0/1530676492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c075f40 msgr2=0x7f282c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c075f40 0x7f282c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 -- 172.21.15.7:0/1530676492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 msgr2=0x7f282c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 0x7f282c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 -- 172.21.15.7:0/1530676492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a20a0 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f282be00640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f282400efd0 tx=0x7f282400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.349+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824003b70 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.350+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2824003d10 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.350+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a2390 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.350+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28240109f0 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.350+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a2890 con 0x7f282c109e10 2024-04-20T21:33:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.351+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c108bd0 con 0x7f282c109e10 2024-04-20T21:33:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.353+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f2824004020 con 0x7f282c109e10 2024-04-20T21:33:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.354+0000 7f2823e00640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 0x7f28100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.355+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f28240d0880 con 0x7f282c109e10 2024-04-20T21:33:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.355+0000 7f282b400640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 0x7f28100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.355+0000 7f282b400640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 0x7f28100a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f281c02c2e0 tx=0x7f281c02c920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.359+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f282409d230 con 0x7f282c109e10 2024-04-20T21:33:05.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:05 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2927487831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:05.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:05 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2927487831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:05.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.567+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f282c062b60 con 0x7f282c109e10 2024-04-20T21:33:05.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.568+0000 7f2823e00640 1 -- 172.21.15.7:0/1530676492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f282409cb30 con 0x7f282c109e10 2024-04-20T21:33:05.568 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 msgr2=0x7f28100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 0x7f28100a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f281c02c2e0 tx=0x7f281c02c920 comp rx=0 tx=0).stop 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 msgr2=0x7f282c1a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f282400efd0 tx=0x7f282400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 shutdown_connections 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f28100a3bb0 0x7f28100a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f282c109e10 0x7f282c1a1a30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f282c075f40 0x7f282c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 --2- 172.21.15.7:0/1530676492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f282c074960 0x7f282c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.571+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 >> 172.21.15.7:0/1530676492 conn(0x7f282c0fd4b0 msgr2=0x7f282c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.572+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 shutdown_connections 2024-04-20T21:33:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.572+0000 7f2832800640 1 -- 172.21.15.7:0/1530676492 wait complete. 2024-04-20T21:33:05.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:05.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:05 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2927487831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:05.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.807+0000 7fb93e200640 1 -- 172.21.15.7:0/4081460316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 msgr2=0x7fb93810c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.807+0000 7fb93e200640 1 --2- 172.21.15.7:0/4081460316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb93810c5f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb930009ca0 tx=0x7fb93002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 -- 172.21.15.7:0/4081460316 shutdown_connections 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 --2- 172.21.15.7:0/4081460316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381053a0 0x7fb938105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 --2- 172.21.15.7:0/4081460316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb93810c5f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 --2- 172.21.15.7:0/4081460316 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381041b0 0x7fb9381045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 -- 172.21.15.7:0/4081460316 >> 172.21.15.7:0/4081460316 conn(0x7fb9380ff720 msgr2=0x7fb938101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 -- 172.21.15.7:0/4081460316 shutdown_connections 2024-04-20T21:33:05.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.808+0000 7fb93e200640 1 -- 172.21.15.7:0/4081460316 wait complete. 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 Processor -- start 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 -- start start 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381041b0 0x7fb93819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 0x7fb93819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810fe40 con 0x7fb938105d60 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810fcc0 con 0x7fb9381041b0 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.809+0000 7fb93e200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb93810ffc0 con 0x7fb9381053a0 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb937e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb937e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41108/0 (socket says 172.21.15.7:41108) 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 0x7fb93819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb937e00640 1 -- 172.21.15.7:0/1639265108 learned_addr learned my addr 172.21.15.7:0/1639265108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb937400640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381041b0 0x7fb93819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 -- 172.21.15.7:0/1639265108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381041b0 msgr2=0x7fb93819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381041b0 0x7fb93819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 -- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 msgr2=0x7fb9381a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 -- 172.21.15.7:0/1639265108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9381a2090 con 0x7fb9381053a0 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb937e00640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:05.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.810+0000 7fb936a00640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 0x7fb93819a9b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb924005fd0 tx=0x7fb92400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.811+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924009c30 con 0x7fb9381053a0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.811+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9381a2380 con 0x7fb9381053a0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.811+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924009dd0 con 0x7fb9381053a0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.811+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924010460 con 0x7fb9381053a0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.811+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a2920 con 0x7fb9381053a0 2024-04-20T21:33:05.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.812+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93810b4d0 con 0x7fb9381053a0 2024-04-20T21:33:05.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.815+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fb924003ef0 con 0x7fb9381053a0 2024-04-20T21:33:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.816+0000 7fb92fe00640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.817+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb9240cfec0 con 0x7fb9381053a0 2024-04-20T21:33:05.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.817+0000 7fb937400640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.817+0000 7fb937400640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb938105200 tx=0x7fb928005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:05.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:05.818+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb924017020 con 0x7fb9381053a0 2024-04-20T21:33:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.021+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb93802d7f0 con 0x7fb9381053a0 2024-04-20T21:33:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.022+0000 7fb92fe00640 1 -- 172.21.15.7:0/1639265108 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fb92409c220 con 0x7fb9381053a0 2024-04-20T21:33:06.022 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:33:06.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.024+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 msgr2=0x7fb91c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb938105200 tx=0x7fb928005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 msgr2=0x7fb93819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 0x7fb93819a9b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb924005fd0 tx=0x7fb92400dde0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 shutdown_connections 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fb91c0a3bb0 0x7fb91c0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb938105d60 0x7fb9381a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9381053a0 0x7fb93819a9b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 --2- 172.21.15.7:0/1639265108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9381041b0 0x7fb93819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 >> 172.21.15.7:0/1639265108 conn(0x7fb9380ff720 msgr2=0x7fb9381086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 shutdown_connections 2024-04-20T21:33:06.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.025+0000 7fb93e200640 1 -- 172.21.15.7:0/1639265108 wait complete. 2024-04-20T21:33:06.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 -- 172.21.15.7:0/2373024556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 msgr2=0x7f1c10104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 --2- 172.21.15.7:0/2373024556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c10104c10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009900 tx=0x7f1c0402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 -- 172.21.15.7:0/2373024556 shutdown_connections 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 --2- 172.21.15.7:0/2373024556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10107a80 0x7f1c10109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 --2- 172.21.15.7:0/2373024556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10105150 0x7f1c10107540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 --2- 172.21.15.7:0/2373024556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c10104c10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 -- 172.21.15.7:0/2373024556 >> 172.21.15.7:0/2373024556 conn(0x7f1c100fc250 msgr2=0x7f1c100fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 -- 172.21.15.7:0/2373024556 shutdown_connections 2024-04-20T21:33:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.269+0000 7f1c15e00640 1 -- 172.21.15.7:0/2373024556 wait complete. 2024-04-20T21:33:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.270+0000 7f1c15e00640 1 Processor -- start 2024-04-20T21:33:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 -- start start 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10105150 0x7f1c1019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 0x7f1c101a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010da90 con 0x7f1c10107a80 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010d910 con 0x7f1c10102820 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.271+0000 7f1c15e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1010dc10 con 0x7f1c10105150 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c15400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 0x7f1c101a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54292/0 (socket says 172.21.15.7:54292) 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 -- 172.21.15.7:0/4035177267 learned_addr learned my addr 172.21.15.7:0/4035177267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10105150 0x7f1c1019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 -- 172.21.15.7:0/4035177267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10105150 msgr2=0x7f1c1019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10105150 0x7f1c1019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 -- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 msgr2=0x7f1c101a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 0x7f1c101a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 -- 172.21.15.7:0/4035177267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c101a1f40 con 0x7f1c10102820 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c15400640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 0x7f1c101a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:06.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.272+0000 7f1c14a00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c0402c8f0 tx=0x7f1c0402c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.273+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04041070 con 0x7f1c10102820 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.273+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c101a21d0 con 0x7f1c10102820 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.273+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c04004450 con 0x7f1c10102820 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.273+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04034440 con 0x7f1c10102820 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.273+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c101a2710 con 0x7f1c10102820 2024-04-20T21:33:06.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.274+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c101095d0 con 0x7f1c10102820 2024-04-20T21:33:06.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.277+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1c04044070 con 0x7f1c10102820 2024-04-20T21:33:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.278+0000 7f1c0d600640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 0x7f1bf80a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.279+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1c040f2340 con 0x7f1c10102820 2024-04-20T21:33:06.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.279+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 0x7f1bf80a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.279+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 0x7f1bf80a6060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c1019b830 tx=0x7f1c00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.280+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1c040be6a0 con 0x7f1c10102820 2024-04-20T21:33:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:06 smithi186 ceph-mon[22157]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:06 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1530676492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:06 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1639265108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.487+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c1002d5d0 con 0x7f1c10102820 2024-04-20T21:33:06.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.488+0000 7f1c0d600640 1 -- 172.21.15.7:0/4035177267 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f1c040c29a0 con 0x7f1c10102820 2024-04-20T21:33:06.488 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 msgr2=0x7f1bf80a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 0x7f1bf80a6060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c1019b830 tx=0x7f1c00005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 msgr2=0x7f1c1019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1c0402c8f0 tx=0x7f1c0402c660 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 shutdown_connections 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f1bf80a3ba0 0x7f1bf80a6060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c10107a80 0x7f1c101a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1c10105150 0x7f1c1019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 --2- 172.21.15.7:0/4035177267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c10102820 0x7f1c1019a320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 >> 172.21.15.7:0/4035177267 conn(0x7f1c100fc250 msgr2=0x7f1c10101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 shutdown_connections 2024-04-20T21:33:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.492+0000 7f1c15e00640 1 -- 172.21.15.7:0/4035177267 wait complete. 2024-04-20T21:33:06.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:06 smithi007 ceph-mon[16867]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:06.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:06 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1530676492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.509 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:06 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1639265108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:06.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:06 smithi016 ceph-mon[21409]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:06.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:06 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1530676492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:06 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1639265108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 -- 172.21.15.7:0/2977722404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 msgr2=0x7f9948078da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 --2- 172.21.15.7:0/2977722404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f9948078da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9940009ca0 tx=0x7f994002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 -- 172.21.15.7:0/2977722404 shutdown_connections 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 --2- 172.21.15.7:0/2977722404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f9948078da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 --2- 172.21.15.7:0/2977722404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 0x7f9948077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 --2- 172.21.15.7:0/2977722404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f9948076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 -- 172.21.15.7:0/2977722404 >> 172.21.15.7:0/2977722404 conn(0x7f99480ff630 msgr2=0x7f9948101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 -- 172.21.15.7:0/2977722404 shutdown_connections 2024-04-20T21:33:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.730+0000 7f994e200640 1 -- 172.21.15.7:0/2977722404 wait complete. 2024-04-20T21:33:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.731+0000 7f994e200640 1 Processor -- start 2024-04-20T21:33:06.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 -- start start 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f994819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 0x7f994819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f99481a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9948114150 con 0x7f9948077b60 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9948113fd0 con 0x7f9948078500 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994e200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f99481142d0 con 0x7f9948076920 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994ce00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f994819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f9947e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 0x7f994819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994ce00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f994819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34298/0 (socket says 172.21.15.7:34298) 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994d800640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f99481a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.732+0000 7f994ce00640 1 -- 172.21.15.7:0/3926318067 learned_addr learned my addr 172.21.15.7:0/3926318067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 -- 172.21.15.7:0/3926318067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 msgr2=0x7f994819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f994819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 -- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 msgr2=0x7f994819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 0x7f994819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 -- 172.21.15.7:0/3926318067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99481a6370 con 0x7f9948078500 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994d800640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f99481a5d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9940037040 tx=0x7f9940032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940041020 con 0x7f9948078500 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99481a6600 con 0x7f9948078500 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9940034cd0 con 0x7f9948078500 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940039040 con 0x7f9948078500 2024-04-20T21:33:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.733+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a6b10 con 0x7f9948078500 2024-04-20T21:33:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.734+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9948110a30 con 0x7f9948078500 2024-04-20T21:33:06.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.737+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f9940044020 con 0x7f9948078500 2024-04-20T21:33:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.738+0000 7f9945600640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 0x7f99300a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.738+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f99400f2550 con 0x7f9948078500 2024-04-20T21:33:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.738+0000 7f994ce00640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 0x7f99300a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.739+0000 7f994ce00640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 0x7f99300a60e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f993c009c90 tx=0x7f993c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.740+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f994004b020 con 0x7f9948078500 2024-04-20T21:33:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.987+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f994802d5d0 con 0x7f9948078500 2024-04-20T21:33:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.988+0000 7f9945600640 1 -- 172.21.15.7:0/3926318067 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7f99400bef90 con 0x7f9948078500 2024-04-20T21:33:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:06.989 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:33:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:33:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 msgr2=0x7f99300a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 0x7f99300a60e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f993c009c90 tx=0x7f993c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 msgr2=0x7f99481a5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f99481a5d00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9940037040 tx=0x7f9940032ea0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 shutdown_connections 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f99300a3c20 0x7f99300a60e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9948078500 0x7f99481a5d00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9948077b60 0x7f994819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 --2- 172.21.15.7:0/3926318067 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9948076920 0x7f994819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.991+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 >> 172.21.15.7:0/3926318067 conn(0x7f99480ff630 msgr2=0x7f9948101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.992+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 shutdown_connections 2024-04-20T21:33:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:06.992+0000 7f994e200640 1 -- 172.21.15.7:0/3926318067 wait complete. 2024-04-20T21:33:07.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.231+0000 7f026ca00640 1 -- 172.21.15.7:0/3898444824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105c50 msgr2=0x7f026810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.231+0000 7f026ca00640 1 --2- 172.21.15.7:0/3898444824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105c50 0x7f026810c4e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0260009900 tx=0x7f026002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 -- 172.21.15.7:0/3898444824 shutdown_connections 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 --2- 172.21.15.7:0/3898444824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105290 0x7f0268105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 --2- 172.21.15.7:0/3898444824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105c50 0x7f026810c4e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 --2- 172.21.15.7:0/3898444824 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f02681044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 -- 172.21.15.7:0/3898444824 >> 172.21.15.7:0/3898444824 conn(0x7f02680ff630 msgr2=0x7f0268101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 -- 172.21.15.7:0/3898444824 shutdown_connections 2024-04-20T21:33:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.232+0000 7f026ca00640 1 -- 172.21.15.7:0/3898444824 wait complete. 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.233+0000 7f026ca00640 1 Processor -- start 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.233+0000 7f026ca00640 1 -- start start 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f0268079d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 0x7f026807a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105c50 0x7f0268076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f026810fd80 con 0x7f0268105290 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f026810fc00 con 0x7f0268105c50 2024-04-20T21:33:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f026ca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f026810ff00 con 0x7f02681040a0 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f025ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 0x7f026807a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f0268079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f025ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 0x7f026807a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41156/0 (socket says 172.21.15.7:41156) 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f025ea00640 1 -- 172.21.15.7:0/3740973174 learned_addr learned my addr 172.21.15.7:0/3740973174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267e00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105c50 0x7f0268076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 -- 172.21.15.7:0/3740973174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105c50 msgr2=0x7f0268076880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105c50 0x7f0268076880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 -- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 msgr2=0x7f026807a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 0x7f026807a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.234+0000 7f0267400640 1 -- 172.21.15.7:0/3740973174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0268077140 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f0267400640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f0268079d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0258005fd0 tx=0x7f025800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258003970 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0258004030 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02580107f0 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02681a6770 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.235+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02681a6bd0 con 0x7f02681040a0 2024-04-20T21:33:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.237+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026810b3c0 con 0x7f02681040a0 2024-04-20T21:33:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.238+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f02580041e0 con 0x7f02681040a0 2024-04-20T21:33:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.240+0000 7f0264c00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 0x7f024c0a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.240+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f02580d0bc0 con 0x7f02681040a0 2024-04-20T21:33:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.240+0000 7f025ea00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 0x7f024c0a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.241+0000 7f025ea00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 0x7f024c0a60e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0268077c90 tx=0x7f0250005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.244+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f025809d570 con 0x7f02681040a0 2024-04-20T21:33:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4035177267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:07.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:07 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3926318067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:07.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.453+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0268062b60 con 0x7f02681040a0 2024-04-20T21:33:07.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.455+0000 7f0264c00640 1 -- 172.21.15.7:0/3740973174 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7f025809ce70 con 0x7f02681040a0 2024-04-20T21:33:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:07.456 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 msgr2=0x7f024c0a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 0x7f024c0a60e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0268077c90 tx=0x7f0250005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 msgr2=0x7f0268079d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f0268079d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0258005fd0 tx=0x7f025800ef90 comp rx=0 tx=0).stop 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 shutdown_connections 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f024c0a3c20 0x7f024c0a60e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0268105c50 0x7f0268076880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0268105290 0x7f026807a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 --2- 172.21.15.7:0/3740973174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02681040a0 0x7f0268079d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 >> 172.21.15.7:0/3740973174 conn(0x7f02680ff630 msgr2=0x7f0268108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.458+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 shutdown_connections 2024-04-20T21:33:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.459+0000 7f026ca00640 1 -- 172.21.15.7:0/3740973174 wait complete. 2024-04-20T21:33:07.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:07.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:07 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4035177267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:07 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3926318067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 -- 172.21.15.7:0/3265659809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 msgr2=0x7fe1c8109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/3265659809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c8109ec0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009be0 tx=0x7fe1c002bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 -- 172.21.15.7:0/3265659809 shutdown_connections 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/3265659809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c8107530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/3265659809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c8109ec0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/3265659809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 -- 172.21.15.7:0/3265659809 >> 172.21.15.7:0/3265659809 conn(0x7fe1c80fc220 msgr2=0x7fe1c80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 -- 172.21.15.7:0/3265659809 shutdown_connections 2024-04-20T21:33:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.697+0000 7fe1c7e00640 1 -- 172.21.15.7:0/3265659809 wait complete. 2024-04-20T21:33:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 Processor -- start 2024-04-20T21:33:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 -- start start 2024-04-20T21:33:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810da80 con 0x7fe1c8102810 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810d900 con 0x7fe1c8105140 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.699+0000 7fe1c7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c810dc00 con 0x7fe1c8107a70 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41164/0 (socket says 172.21.15.7:41164) 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6a00640 1 -- 172.21.15.7:0/1915554785 learned_addr learned my addr 172.21.15.7:0/1915554785 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c7400640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 -- 172.21.15.7:0/1915554785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 msgr2=0x7fe1c819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 -- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 msgr2=0x7fe1c8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 -- 172.21.15.7:0/1915554785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c819fc90 con 0x7fe1c8105140 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.700+0000 7fe1c6000640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c81985b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9c0 tx=0x7fe1b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.701+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400ccc0 con 0x7fe1c8105140 2024-04-20T21:33:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.701+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b4005020 con 0x7fe1c8105140 2024-04-20T21:33:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.701+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c819ff80 con 0x7fe1c8105140 2024-04-20T21:33:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.701+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4004a00 con 0x7fe1c8105140 2024-04-20T21:33:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.701+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c81a03f0 con 0x7fe1c8105140 2024-04-20T21:33:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.702+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c80fdd10 con 0x7fe1c8105140 2024-04-20T21:33:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.705+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fe1b40051d0 con 0x7fe1c8105140 2024-04-20T21:33:07.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.706+0000 7fe1bf400640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 0x7fe1ac0a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:07.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.707+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe1b40d03a0 con 0x7fe1c8105140 2024-04-20T21:33:07.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.707+0000 7fe1c6a00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 0x7fe1ac0a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:07.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.707+0000 7fe1c6a00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 0x7fe1ac0a6060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80058c0 tx=0x7fe1b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:07.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.711+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe1b409cd50 con 0x7fe1c8105140 2024-04-20T21:33:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:07 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4035177267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:07 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3926318067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.919+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1c8062c00 con 0x7fe1c8105140 2024-04-20T21:33:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.919+0000 7fe1bf400640 1 -- 172.21.15.7:0/1915554785 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe1b409c650 con 0x7fe1c8105140 2024-04-20T21:33:07.919 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 msgr2=0x7fe1ac0a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 0x7fe1ac0a6060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80058c0 tx=0x7fe1b8009260 comp rx=0 tx=0).stop 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 msgr2=0x7fe1c81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c81985b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9c0 tx=0x7fe1b400ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 shutdown_connections 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe1ac0a3ba0 0x7fe1ac0a6060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c8105140 0x7fe1c81985b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1c8107a70 0x7fe1c819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 --2- 172.21.15.7:0/1915554785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c8102810 0x7fe1c8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 >> 172.21.15.7:0/1915554785 conn(0x7fe1c80fc220 msgr2=0x7fe1c8103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 shutdown_connections 2024-04-20T21:33:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:07.922+0000 7fe1c7e00640 1 -- 172.21.15.7:0/1915554785 wait complete. 2024-04-20T21:33:07.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 -- 172.21.15.7:0/1870848476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 msgr2=0x7fc35c105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 --2- 172.21.15.7:0/1870848476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c105650 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc344009900 tx=0x7fc34402bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:08.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 -- 172.21.15.7:0/1870848476 shutdown_connections 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 --2- 172.21.15.7:0/1870848476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 0x7fc35c10c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 --2- 172.21.15.7:0/1870848476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c105650 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 --2- 172.21.15.7:0/1870848476 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 0x7fc35c104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.160+0000 7fc35be00640 1 -- 172.21.15.7:0/1870848476 >> 172.21.15.7:0/1870848476 conn(0x7fc35c0ff570 msgr2=0x7fc35c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.161+0000 7fc35be00640 1 -- 172.21.15.7:0/1870848476 shutdown_connections 2024-04-20T21:33:08.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.161+0000 7fc35be00640 1 -- 172.21.15.7:0/1870848476 wait complete. 2024-04-20T21:33:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 Processor -- start 2024-04-20T21:33:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 -- start start 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 0x7fc35c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 0x7fc35c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c10fcc0 con 0x7fc35c1051d0 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c10fb40 con 0x7fc35c105b90 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc35be00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc35c10fe40 con 0x7fc35c103fe0 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc352000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc352000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41182/0 (socket says 172.21.15.7:41182) 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.162+0000 7fc352000640 1 -- 172.21.15.7:0/4096523442 learned_addr learned my addr 172.21.15.7:0/4096523442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc35aa00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 0x7fc35c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc35b400640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 0x7fc35c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 -- 172.21.15.7:0/4096523442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 msgr2=0x7fc35c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 0x7fc35c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 -- 172.21.15.7:0/4096523442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 msgr2=0x7fc35c1a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 0x7fc35c1a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 -- 172.21.15.7:0/4096523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35c1a1f10 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc352000640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc34402c260 tx=0x7fc34402c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.163+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344041070 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.164+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc344004450 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.164+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc35c1a21a0 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.164+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344034440 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.164+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35c1a26f0 con 0x7fc35c1051d0 2024-04-20T21:33:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.165+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35c10b300 con 0x7fc35c1051d0 2024-04-20T21:33:08.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.167+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc344044070 con 0x7fc35c1051d0 2024-04-20T21:33:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.168+0000 7fc353e00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 0x7fc3400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.169+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc3440f1ed0 con 0x7fc35c1051d0 2024-04-20T21:33:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.169+0000 7fc35aa00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 0x7fc3400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.170+0000 7fc35aa00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 0x7fc3400a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0095f0 tx=0x7fc34c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.173+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc3440be230 con 0x7fc35c1051d0 2024-04-20T21:33:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.378+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc35c062b60 con 0x7fc35c1051d0 2024-04-20T21:33:08.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.379+0000 7fc353e00640 1 -- 172.21.15.7:0/4096523442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fc3440c2530 con 0x7fc35c1051d0 2024-04-20T21:33:08.379 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:33:08.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.381+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 msgr2=0x7fc3400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.381+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 0x7fc3400a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0095f0 tx=0x7fc34c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.381+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 msgr2=0x7fc35c19a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.381+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc34402c260 tx=0x7fc34402c670 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.381+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 shutdown_connections 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc3400a3bb0 0x7fc3400a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc35c105b90 0x7fc35c1a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc35c1051d0 0x7fc35c19a830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 --2- 172.21.15.7:0/4096523442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc35c103fe0 0x7fc35c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 >> 172.21.15.7:0/4096523442 conn(0x7fc35c0ff570 msgr2=0x7fc35c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 shutdown_connections 2024-04-20T21:33:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.382+0000 7fc35be00640 1 -- 172.21.15.7:0/4096523442 wait complete. 2024-04-20T21:33:08.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:08.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:08 smithi007 ceph-mon[16867]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:08.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3740973174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:08.477 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1915554785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 -- 172.21.15.7:0/1337025912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c1041b0 msgr2=0x7ff65c1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 --2- 172.21.15.7:0/1337025912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c1041b0 0x7ff65c1045d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009900 tx=0x7ff64c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 -- 172.21.15.7:0/1337025912 shutdown_connections 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 --2- 172.21.15.7:0/1337025912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c105d60 0x7ff65c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 --2- 172.21.15.7:0/1337025912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1053a0 0x7ff65c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 --2- 172.21.15.7:0/1337025912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c1041b0 0x7ff65c1045d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 -- 172.21.15.7:0/1337025912 >> 172.21.15.7:0/1337025912 conn(0x7ff65c0ff720 msgr2=0x7ff65c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 -- 172.21.15.7:0/1337025912 shutdown_connections 2024-04-20T21:33:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.625+0000 7ff663200640 1 -- 172.21.15.7:0/1337025912 wait complete. 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 Processor -- start 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 -- start start 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 0x7ff65c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10fe90 con 0x7ff65c1041b0 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c10fd10 con 0x7ff65c105d60 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.626+0000 7ff663200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff65c110010 con 0x7ff65c1053a0 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff661200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff661200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54390/0 (socket says 172.21.15.7:54390) 2024-04-20T21:33:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 0x7ff65c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff661200640 1 -- 172.21.15.7:0/3668352182 learned_addr learned my addr 172.21.15.7:0/3668352182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41198/0 (socket says 172.21.15.7:41198) 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 -- 172.21.15.7:0/3668352182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 msgr2=0x7ff65c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 0x7ff65c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 -- 172.21.15.7:0/3668352182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 msgr2=0x7ff65c1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65be00640 1 -- 172.21.15.7:0/3668352182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c1a20c0 con 0x7ff65c1041b0 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff65b400640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 0x7ff65c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.627+0000 7ff661200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff65be00640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009230 tx=0x7ff64c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c043020 con 0x7ff65c1041b0 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff64c0054e0 con 0x7ff65c1041b0 2024-04-20T21:33:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c003d80 con 0x7ff65c1041b0 2024-04-20T21:33:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65c1a2350 con 0x7ff65c1041b0 2024-04-20T21:33:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.628+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65c1a28c0 con 0x7ff65c1041b0 2024-04-20T21:33:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.629+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff65c10b4d0 con 0x7ff65c1041b0 2024-04-20T21:33:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.631+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7ff64c046020 con 0x7ff65c1041b0 2024-04-20T21:33:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.633+0000 7ff658c00640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 0x7ff6400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.633+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff64c0f24f0 con 0x7ff65c1041b0 2024-04-20T21:33:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.633+0000 7ff65b400640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 0x7ff6400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.634+0000 7ff65b400640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 0x7ff6400a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff65c19b9b0 tx=0x7ff64800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:08.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.638+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff64c04d020 con 0x7ff65c1041b0 2024-04-20T21:33:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:08 smithi016 ceph-mon[21409]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3740973174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1915554785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:08 smithi186 ceph-mon[22157]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3740973174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1915554785' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.843+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff65c062b60 con 0x7ff65c1041b0 2024-04-20T21:33:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.844+0000 7ff658c00640 1 -- 172.21.15.7:0/3668352182 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7ff64c0beea0 con 0x7ff65c1041b0 2024-04-20T21:33:08.844 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:33:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 msgr2=0x7ff6400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 0x7ff6400a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff65c19b9b0 tx=0x7ff64800b410 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 msgr2=0x7ff65c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009230 tx=0x7ff64c005dd0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 shutdown_connections 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ff6400a3c30 0x7ff6400a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff65c105d60 0x7ff65c1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff65c1053a0 0x7ff65c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 --2- 172.21.15.7:0/3668352182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff65c1041b0 0x7ff65c19a4a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 >> 172.21.15.7:0/3668352182 conn(0x7ff65c0ff720 msgr2=0x7ff65c1086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:08.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 shutdown_connections 2024-04-20T21:33:08.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:08.847+0000 7ff663200640 1 -- 172.21.15.7:0/3668352182 wait complete. 2024-04-20T21:33:08.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:09.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.078+0000 7ffa91c00640 1 -- 172.21.15.7:0/1255861548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105430 msgr2=0x7ffa8c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.078+0000 7ffa91c00640 1 --2- 172.21.15.7:0/1255861548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105430 0x7ffa8c1058b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009900 tx=0x7ffa7402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 -- 172.21.15.7:0/1255861548 shutdown_connections 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 --2- 172.21.15.7:0/1255861548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105430 0x7ffa8c1058b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 --2- 172.21.15.7:0/1255861548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105df0 0x7ffa8c10c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 --2- 172.21.15.7:0/1255861548 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 -- 172.21.15.7:0/1255861548 >> 172.21.15.7:0/1255861548 conn(0x7ffa8c0ff7b0 msgr2=0x7ffa8c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 -- 172.21.15.7:0/1255861548 shutdown_connections 2024-04-20T21:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.079+0000 7ffa91c00640 1 -- 172.21.15.7:0/1255861548 wait complete. 2024-04-20T21:33:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.080+0000 7ffa91c00640 1 Processor -- start 2024-04-20T21:33:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.080+0000 7ffa91c00640 1 -- start start 2024-04-20T21:33:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 0x7ffa8c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105df0 0x7ffa8c1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c10feb0 con 0x7ffa8c105430 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c10fd30 con 0x7ffa8c105df0 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa91c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c110030 con 0x7ffa8c104240 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa8aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa8aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34372/0 (socket says 172.21.15.7:34372) 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa8aa00640 1 -- 172.21.15.7:0/461447277 learned_addr learned my addr 172.21.15.7:0/461447277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa8a000640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 0x7ffa8c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.081+0000 7ffa8b400640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105df0 0x7ffa8c1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 -- 172.21.15.7:0/461447277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105df0 msgr2=0x7ffa8c1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105df0 0x7ffa8c1a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 -- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 msgr2=0x7ffa8c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 0x7ffa8c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 -- 172.21.15.7:0/461447277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c1a20d0 con 0x7ffa8c104240 2024-04-20T21:33:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8a000640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 0x7ffa8c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa8aa00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9c0 tx=0x7ffa8400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400cd80 con 0x7ffa8c104240 2024-04-20T21:33:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa840035f0 con 0x7ffa8c104240 2024-04-20T21:33:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c1a23c0 con 0x7ffa8c104240 2024-04-20T21:33:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.082+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84010690 con 0x7ffa8c104240 2024-04-20T21:33:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.083+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a28e0 con 0x7ffa8c104240 2024-04-20T21:33:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.084+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8c10b500 con 0x7ffa8c104240 2024-04-20T21:33:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.086+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7ffa840037a0 con 0x7ffa8c104240 2024-04-20T21:33:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.087+0000 7ffa83400640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 0x7ffa700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.088+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ffa840d0760 con 0x7ffa8c104240 2024-04-20T21:33:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.088+0000 7ffa8a000640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 0x7ffa700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.089+0000 7ffa8a000640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 0x7ffa700a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa7402c250 tx=0x7ffa74005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.091+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffa8409d170 con 0x7ffa8c104240 2024-04-20T21:33:09.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.344+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa8c02d5d0 con 0x7ffa8c104240 2024-04-20T21:33:09.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.345+0000 7ffa83400640 1 -- 172.21.15.7:0/461447277 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11869 (secure 0 0 0) 0x7ffa8409ca70 con 0x7ffa8c104240 2024-04-20T21:33:09.345 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:09.345 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:30:58.437063+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:33:09.345 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 msgr2=0x7ffa700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 0x7ffa700a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa7402c250 tx=0x7ffa74005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 msgr2=0x7ffa8c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e9c0 tx=0x7ffa8400ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 shutdown_connections 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7ffa700a3c30 0x7ffa700a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa8c105df0 0x7ffa8c1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa8c105430 0x7ffa8c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 --2- 172.21.15.7:0/461447277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8c104240 0x7ffa8c19a4b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 >> 172.21.15.7:0/461447277 conn(0x7ffa8c0ff7b0 msgr2=0x7ffa8c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 shutdown_connections 2024-04-20T21:33:09.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.348+0000 7ffa91c00640 1 -- 172.21.15.7:0/461447277 wait complete. 2024-04-20T21:33:09.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:09.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:09 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/4096523442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:09 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3668352182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 -- 172.21.15.7:0/3519775849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 msgr2=0x7fc454106120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 --2- 172.21.15.7:0/3519775849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc454106120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009ca0 tx=0x7fc44c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 -- 172.21.15.7:0/3519775849 shutdown_connections 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 --2- 172.21.15.7:0/3519775849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc454106120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 --2- 172.21.15.7:0/3519775849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100ec0 0x7fc454101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 --2- 172.21.15.7:0/3519775849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100160 0x7fc454100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 -- 172.21.15.7:0/3519775849 >> 172.21.15.7:0/3519775849 conn(0x7fc4540fbad0 msgr2=0x7fc4540fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 -- 172.21.15.7:0/3519775849 shutdown_connections 2024-04-20T21:33:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.584+0000 7fc453e00640 1 -- 172.21.15.7:0/3519775849 wait complete. 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 Processor -- start 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 -- start start 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100160 0x7fc45419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100ec0 0x7fc45419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc454114190 con 0x7fc454100ec0 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc454114010 con 0x7fc454101880 2024-04-20T21:33:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.585+0000 7fc453e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc454114310 con 0x7fc454100160 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc44be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100ec0 0x7fc45419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54416/0 (socket says 172.21.15.7:54416) 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 -- 172.21.15.7:0/2514973197 learned_addr learned my addr 172.21.15.7:0/2514973197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc452a00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100160 0x7fc45419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 -- 172.21.15.7:0/2514973197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100160 msgr2=0x7fc45419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100160 0x7fc45419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 -- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100ec0 msgr2=0x7fc45419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100ec0 0x7fc45419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 -- 172.21.15.7:0/2514973197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4541a6390 con 0x7fc454101880 2024-04-20T21:33:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.586+0000 7fc453400640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02c630 tx=0x7fc44c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.587+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c041020 con 0x7fc454101880 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.587+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4541a6620 con 0x7fc454101880 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.587+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc44c034cd0 con 0x7fc454101880 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.587+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c039040 con 0x7fc454101880 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.587+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4541a6b60 con 0x7fc454101880 2024-04-20T21:33:09.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.588+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc454104f30 con 0x7fc454101880 2024-04-20T21:33:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.590+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fc44c044020 con 0x7fc454101880 2024-04-20T21:33:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.591+0000 7fc458a00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 0x7fc4380a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.592+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc44c0f2210 con 0x7fc454101880 2024-04-20T21:33:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.592+0000 7fc452a00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 0x7fc4380a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.593+0000 7fc452a00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 0x7fc4380a6060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc444009bd0 tx=0x7fc444005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.596+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc44c04b020 con 0x7fc454101880 2024-04-20T21:33:09.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:09 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/4096523442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:09 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3668352182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:09 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/4096523442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:09 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3668352182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.804+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc454062b60 con 0x7fc454101880 2024-04-20T21:33:09.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.806+0000 7fc458a00640 1 -- 172.21.15.7:0/2514973197 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22488 (secure 0 0 0) 0x7fc44c0bec50 con 0x7fc454101880 2024-04-20T21:33:09.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:09.807 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:31:50.134558+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 msgr2=0x7fc4380a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 0x7fc4380a6060 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc444009bd0 tx=0x7fc444005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 msgr2=0x7fc4541a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc44c02c630 tx=0x7fc44c032ea0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 shutdown_connections 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc4380a3ba0 0x7fc4380a6060 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc454101880 0x7fc4541a5d20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc454100ec0 0x7fc45419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 --2- 172.21.15.7:0/2514973197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc454100160 0x7fc45419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:09.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 >> 172.21.15.7:0/2514973197 conn(0x7fc4540fbad0 msgr2=0x7fc454102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:09.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.809+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 shutdown_connections 2024-04-20T21:33:09.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:09.810+0000 7fc453e00640 1 -- 172.21.15.7:0/2514973197 wait complete. 2024-04-20T21:33:09.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-20T21:30:52.886616+0000', 'modified': '2024-04-20T21:30:58.437063+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24575, 'mds_1': 24559, 'mds_2': 24535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6835/2330254135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2330254135}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2330254135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.16:6837/4051126294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6836', 'nonce': 4051126294}, {'type': 'v1', 'addr': '172.21.15.16:6837', 'nonce': 4051126294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24523': {'gid': 24523, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/2162825015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 2162825015}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 2162825015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.186:6837/2315081064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 2315081064}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 2315081064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24559': {'gid': 24559, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.186:6839/2448649165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2448649165}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2448649165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6839/518118418', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6838', 'nonce': 518118418}, {'type': 'v1', 'addr': '172.21.15.16:6839', 'nonce': 518118418}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24575, 'qdb_cluster': [24535, 24559, 24575]} 2024-04-20T21:33:09.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-20T21:33:09.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:33:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 -- 172.21.15.7:0/1603648522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244105b90 msgr2=0x7fe24410c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 --2- 172.21.15.7:0/1603648522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244105b90 0x7fe24410c420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009be0 tx=0x7fe23c02bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 -- 172.21.15.7:0/1603648522 shutdown_connections 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 --2- 172.21.15.7:0/1603648522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2441051d0 0x7fe244105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 --2- 172.21.15.7:0/1603648522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244105b90 0x7fe24410c420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 --2- 172.21.15.7:0/1603648522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe244103fe0 0x7fe244104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 -- 172.21.15.7:0/1603648522 >> 172.21.15.7:0/1603648522 conn(0x7fe2440ff570 msgr2=0x7fe244101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 -- 172.21.15.7:0/1603648522 shutdown_connections 2024-04-20T21:33:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.052+0000 7fe249600640 1 -- 172.21.15.7:0/1603648522 wait complete. 2024-04-20T21:33:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.053+0000 7fe249600640 1 Processor -- start 2024-04-20T21:33:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 -- start start 2024-04-20T21:33:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 0x7fe24419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 0x7fe24419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 0x7fe2441a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fc80 con 0x7fe2441051d0 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fb00 con 0x7fe244105b90 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe249600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe24410fe00 con 0x7fe244103fe0 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe243400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 0x7fe24419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe243400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 0x7fe24419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41266/0 (socket says 172.21.15.7:41266) 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe243400640 1 -- 172.21.15.7:0/3526572278 learned_addr learned my addr 172.21.15.7:0/3526572278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe248c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 0x7fe2441a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.054+0000 7fe243e00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 0x7fe24419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 -- 172.21.15.7:0/3526572278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 msgr2=0x7fe24419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 0x7fe24419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 -- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 msgr2=0x7fe24419a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 0x7fe24419a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 -- 172.21.15.7:0/3526572278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2441a1f00 con 0x7fe244105b90 2024-04-20T21:33:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe243e00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 0x7fe24419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe248c00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 0x7fe2441a1890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe23c02c430 tx=0x7fe23c02cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe240c00640 1 -- 172.21.15.7:0/3526572278 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c043020 con 0x7fe244105b90 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2441a2190 con 0x7fe244105b90 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe240c00640 1 -- 172.21.15.7:0/3526572278 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23c003e50 con 0x7fe244105b90 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.055+0000 7fe240c00640 1 -- 172.21.15.7:0/3526572278 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c034430 con 0x7fe244105b90 2024-04-20T21:33:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.056+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a26e0 con 0x7fe244105b90 2024-04-20T21:33:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.058+0000 7fe24d5fd740 1 Processor -- start 2024-04-20T21:33:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.059+0000 7fe240c00640 1 -- 172.21.15.7:0/3526572278 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fe23c046020 con 0x7fe244105b90 2024-04-20T21:33:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.059+0000 7fe24d5fd740 1 -- start start 2024-04-20T21:33:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.059+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a6b40 0x558fd05a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd050d9d0 con 0x558fd05ac6a0 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558fd04b4270 con 0x558fd05a6b40 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558fd0509990 con 0x558fd05a7530 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a6b40 0x558fd05a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34404/0 (socket says 172.21.15.7:34404) 2024-04-20T21:33:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 -- 172.21.15.7:0/1071762963 learned_addr learned my addr 172.21.15.7:0/1071762963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 -- 172.21.15.7:0/1071762963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a6b40 msgr2=0x558fd05a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a6b40 0x558fd05a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 -- 172.21.15.7:0/1071762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 msgr2=0x558fd05aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.060+0000 7fe243e00640 1 -- 172.21.15.7:0/1071762963 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd02202d0 con 0x558fd05a7530 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe243e00640 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2340114f0 tx=0x7fe2340119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9274893df8ceb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe240c00640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 0x7fe2280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe240c00640 1 -- 172.21.15.7:0/3526572278 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe23c0f2960 con 0x7fe244105b90 2024-04-20T21:33:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe243400640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 0x7fe2280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe239200640 1 -- 172.21.15.7:0/1071762963 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234011310 con 0x558fd05a7530 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe239200640 1 -- 172.21.15.7:0/1071762963 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe234005500 con 0x558fd05a7530 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.061+0000 7fe239200640 1 -- 172.21.15.7:0/1071762963 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400f820 con 0x558fd05a7530 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- 172.21.15.7:0/1071762963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 msgr2=0x558fd05ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe243400640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 0x7fe2280a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x558fd05a7d90 tx=0x7fe230004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2340114f0 tx=0x7fe2340119b0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- 172.21.15.7:0/1071762963 shutdown_connections 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a7530 0x558fd05ac030 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/1071762963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a6b40 0x558fd05a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- 172.21.15.7:0/1071762963 >> 172.21.15.7:0/1071762963 conn(0x558fd05a20b0 msgr2=0x558fd05a24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- 172.21.15.7:0/1071762963 shutdown_connections 2024-04-20T21:33:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- 172.21.15.7:0/1071762963 wait complete. 2024-04-20T21:33:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.062+0000 7fe24d5fd740 1 -- start start 2024-04-20T21:33:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 Processor -- start 2024-04-20T21:33:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd05af650 con 0x558fd05ac6a0 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558fd04c75e0 con 0x558fd05a7530 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558fd05af7d0 con 0x558fd05a6b40 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe24d5fd740 10 client.0 _ll_register_callbacks cb 0x558fd021fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe243e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41280/0 (socket says 172.21.15.7:41280) 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe243400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.063+0000 7fe248c00640 1 -- 172.21.15.7:0/3142000152 learned_addr learned my addr 172.21.15.7:0/3142000152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 -- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 msgr2=0x558fd05b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 -- 172.21.15.7:0/3142000152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 msgr2=0x558fd05b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 -- 172.21.15.7:0/3142000152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd02ea6b0 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe243e00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe248c00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0beba0 tx=0x7fe23c0beaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eb124a98500f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe22d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0bec10 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.064+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe23c0df070 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe234005500 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe23c0d8170 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 10 client.15039 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558fd05bb9b0 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe234005500 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7fe23c004b40 con 0x558fd05ac6a0 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 10 client.15039 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:33:10.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 20 client.15039 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd05be0c0 0x558fd05bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 4 client.15039 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.065+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fd051b3f0 con 0x558fd05be0c0 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.066+0000 7fe243e00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd05be0c0 0x558fd05bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.066+0000 7fe243e00640 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd05be0c0 0x558fd05bf530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c06352c77015185a server_cookie=f52336a8e12a1570 in_seq=0 out_seq=0 2024-04-20T21:33:10.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.066+0000 7fe22d600640 10 client.15039 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.067+0000 7fe22d600640 1 -- 172.21.15.7:0/3142000152 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fd051b3f0 con 0x558fd05be0c0 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.067+0000 7fe22d600640 10 client.15039 handle_command_reply: tid=0 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.067+0000 7fe24d5fd740 1 client.15039 shutdown 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 msgr2=0x558fd05b8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0beba0 tx=0x7fe23c0beaa0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 shutdown_connections 2024-04-20T21:33:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd05be0c0 0x558fd05bf530 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b8640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/3142000152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 >> 172.21.15.7:0/3142000152 conn(0x558fd05a20b0 msgr2=0x558fd05a13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe22d600640 0 client.15039 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 shutdown_connections 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 1 -- 172.21.15.7:0/3142000152 wait complete. 2024-04-20T21:33:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.068+0000 7fe24d5fd740 20 client.15039 trim_cache size 0 max 16384 2024-04-20T21:33:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 Processor -- start 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 -- start start 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05afba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd05bd870 con 0x558fd05ac6a0 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558fd05bdf40 con 0x558fd05a7530 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.088+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558fd05be610 con 0x558fd05a6b40 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54452/0 (socket says 172.21.15.7:54452) 2024-04-20T21:33:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243400640 1 -- 172.21.15.7:0/4121941704 learned_addr learned my addr 172.21.15.7:0/4121941704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41284/0 (socket says 172.21.15.7:41284) 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 -- 172.21.15.7:0/4121941704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 msgr2=0x558fd05a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 -- 172.21.15.7:0/4121941704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 msgr2=0x558fd05bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243e00640 1 -- 172.21.15.7:0/4121941704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd05ac3f0 con 0x558fd05a6b40 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe243400640 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.089+0000 7fe248c00640 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.090+0000 7fe243e00640 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05afba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558fd05a3020 tx=0x7fe234009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d026ef6017e514b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.090+0000 7fe22fe00640 1 -- 172.21.15.7:0/4121941704 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234032070 con 0x558fd05a6b40 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.090+0000 7fe22fe00640 1 -- 172.21.15.7:0/4121941704 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23402dbc0 con 0x558fd05a6b40 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.090+0000 7fe22fe00640 1 -- 172.21.15.7:0/4121941704 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23402d7c0 con 0x558fd05a6b40 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- 172.21.15.7:0/4121941704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 msgr2=0x558fd05afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05afba0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x558fd05a3020 tx=0x7fe234009260 comp rx=0 tx=0).stop 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- 172.21.15.7:0/4121941704 shutdown_connections 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/4121941704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05afba0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- 172.21.15.7:0/4121941704 >> 172.21.15.7:0/4121941704 conn(0x558fd05a20b0 msgr2=0x558fd05a7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- 172.21.15.7:0/4121941704 shutdown_connections 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- 172.21.15.7:0/4121941704 wait complete. 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.091+0000 7fe24d5fd740 1 -- start start 2024-04-20T21:33:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 Processor -- start 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558fd05bf3b0 con 0x558fd05ac6a0 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558fd05bece0 con 0x558fd05a7530 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558fd05bfa80 con 0x558fd05a6b40 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe24d5fd740 10 client.0 _ll_register_callbacks cb 0x7fe23c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe248c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41288/0 (socket says 172.21.15.7:41288) 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe248c00640 1 -- 172.21.15.7:0/2738517760 learned_addr learned my addr 172.21.15.7:0/2738517760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe243e00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.092+0000 7fe243400640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 -- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 msgr2=0x558fd05b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 -- 172.21.15.7:0/2738517760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 msgr2=0x558fd05b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 -- 172.21.15.7:0/2738517760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd05b6e50 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe243e00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe248c00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe23c004eb0 tx=0x7fe23c0beb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496eaaa8db5321c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe22d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c088060 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe23c0df0b0 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.093+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23402dbc0 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe23c0dd040 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 10 client.15045 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x558fd0624130 con 0x558fd05ac6a0 2024-04-20T21:33:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23402dbc0 con 0x558fd05ac6a0 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7fe23c0ba960 con 0x558fd05ac6a0 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 10 client.15045 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 20 client.15045 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd0628990 0x558fd06276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 4 client.15045 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "status"}) -- 0x558fd0515260 con 0x558fd0628990 2024-04-20T21:33:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.094+0000 7fe243e00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd0628990 0x558fd06276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.095+0000 7fe243e00640 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd0628990 0x558fd06276a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d37687f2c7f8e1b server_cookie=3bf8ee1537ca9388 in_seq=0 out_seq=0 2024-04-20T21:33:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.095+0000 7fe22d600640 10 client.15045 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.095+0000 7fe22d600640 1 -- 172.21.15.7:0/2738517760 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x558fd0515260 con 0x558fd0628990 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.095+0000 7fe22d600640 10 client.15045 handle_command_reply: tid=0 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.095+0000 7fe24d5fd740 1 client.15045 shutdown 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 msgr2=0x558fd05b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe23c004eb0 tx=0x7fe23c0beb10 comp rx=0 tx=0).stop 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 shutdown_connections 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x558fd0628990 0x558fd06276a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558fd05a7530 0x558fd05b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558fd05ac6a0 0x558fd05b6860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 --2- 172.21.15.7:0/2738517760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558fd05a6b40 0x558fd05b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe22d600640 0 client.15045 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 >> 172.21.15.7:0/2738517760 conn(0x558fd05a20b0 msgr2=0x558fd05afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 shutdown_connections 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 1 -- 172.21.15.7:0/2738517760 wait complete. 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.096+0000 7fe24d5fd740 20 client.15045 trim_cache size 0 max 16384 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24535, 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 132.14810368299999, 2024-04-20T21:33:10.101 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:33:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:33:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:33:10.102 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 160.76196285 2024-04-20T21:33:10.102 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:33:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 msgr2=0x7fe2280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 0x7fe2280a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x558fd05a7d90 tx=0x7fe230004730 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 msgr2=0x7fe2441a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 0x7fe2441a1890 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe23c02c430 tx=0x7fe23c02cb90 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 shutdown_connections 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fe2280a3bb0 0x7fe2280a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe244105b90 0x7fe2441a1890 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2441051d0 0x7fe24419a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 --2- 172.21.15.7:0/3526572278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe244103fe0 0x7fe24419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 >> 172.21.15.7:0/3526572278 conn(0x7fe2440ff570 msgr2=0x7fe244108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 shutdown_connections 2024-04-20T21:33:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.097+0000 7fe249600640 1 -- 172.21.15.7:0/3526572278 wait complete. 2024-04-20T21:33:10.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-20T21:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 -- 172.21.15.7:0/2523420990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974076920 msgr2=0x7f1974076d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 --2- 172.21.15.7:0/2523420990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974076920 0x7f1974076d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1964009be0 tx=0x7f196402bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 -- 172.21.15.7:0/2523420990 shutdown_connections 2024-04-20T21:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 --2- 172.21.15.7:0/2523420990 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974078500 0x7f1974078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 --2- 172.21.15.7:0/2523420990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974077b60 0x7f1974077fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 --2- 172.21.15.7:0/2523420990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974076920 0x7f1974076d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 -- 172.21.15.7:0/2523420990 >> 172.21.15.7:0/2523420990 conn(0x7f19740ff570 msgr2=0x7f1974101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 -- 172.21.15.7:0/2523420990 shutdown_connections 2024-04-20T21:33:10.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.330+0000 7f1979000640 1 -- 172.21.15.7:0/2523420990 wait complete. 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.331+0000 7f1979000640 1 Processor -- start 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.331+0000 7f1979000640 1 -- start start 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 0x7f197419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 0x7f19741a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1974114070 con 0x7f1974078500 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1974113ef0 con 0x7f1974077b60 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1979000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f19741141f0 con 0x7f1974076920 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1973400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 0x7f19741a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1973400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34438/0 (socket says 172.21.15.7:34438) 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 0x7f19741a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41304/0 (socket says 172.21.15.7:41304) 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1973400640 1 -- 172.21.15.7:0/2491593066 learned_addr learned my addr 172.21.15.7:0/2491593066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.332+0000 7f1972a00640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 0x7f197419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 -- 172.21.15.7:0/2491593066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 msgr2=0x7f197419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 0x7f197419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 -- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 msgr2=0x7f19741a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 0x7f19741a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 -- 172.21.15.7:0/2491593066 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741a62d0 con 0x7f1974076920 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1972a00640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 0x7f197419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1973400640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1964000f80 tx=0x7f19640337b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f196be00640 1 -- 172.21.15.7:0/2491593066 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964043020 con 0x7f1974076920 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f196be00640 1 -- 172.21.15.7:0/2491593066 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f196402ca60 con 0x7f1974076920 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741a6560 con 0x7f1974076920 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.333+0000 7f196be00640 1 -- 172.21.15.7:0/2491593066 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964034690 con 0x7f1974076920 2024-04-20T21:33:10.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.334+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741a6ab0 con 0x7f1974076920 2024-04-20T21:33:10.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f196be00640 1 -- 172.21.15.7:0/2491593066 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f1964046020 con 0x7f1974076920 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 Processor -- start 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 -- start start 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f54346030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54348b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f542a79d0 con 0x560f54340b40 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f5424e270 con 0x560f54341530 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.337+0000 7f197d3fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560f542a3990 con 0x560f543466a0 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41306/0 (socket says 172.21.15.7:41306) 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 -- 172.21.15.7:0/2486231042 learned_addr learned my addr 172.21.15.7:0/2486231042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973400640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f54346030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1972a00640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54348b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 -- 172.21.15.7:0/2486231042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 msgr2=0x560f54348b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54348b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 -- 172.21.15.7:0/2486231042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 msgr2=0x560f54346030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f54346030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 -- 172.21.15.7:0/2486231042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f53fba2d0 con 0x560f54340b40 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973400640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f54346030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.338+0000 7f1973e00640 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ea761d383a08ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f196be00640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f1968c00640 1 -- 172.21.15.7:0/2486231042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c01e070 con 0x560f54340b40 2024-04-20T21:33:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f1968c00640 1 -- 172.21.15.7:0/2486231042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f196c009e50 con 0x560f54340b40 2024-04-20T21:33:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f1968c00640 1 -- 172.21.15.7:0/2486231042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00eac0 con 0x560f54340b40 2024-04-20T21:33:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f196be00640 1 -- 172.21.15.7:0/2491593066 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f19640f2a50 con 0x7f1974076920 2024-04-20T21:33:10.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.339+0000 7f1973400640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- 172.21.15.7:0/2486231042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 msgr2=0x560f54340f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c011810 comp rx=0 tx=0).stop 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- 172.21.15.7:0/2486231042 shutdown_connections 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f54346030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54348b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2486231042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54340b40 0x560f54340f60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- 172.21.15.7:0/2486231042 >> 172.21.15.7:0/2486231042 conn(0x560f5433c0b0 msgr2=0x560f5433c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f1973400640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1964034fd0 tx=0x7f19640f8530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- 172.21.15.7:0/2486231042 shutdown_connections 2024-04-20T21:33:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- 172.21.15.7:0/2486231042 wait complete. 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.340+0000 7f197d3fc740 1 -- start start 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 Processor -- start 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54352630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f54349650 con 0x560f54341530 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f542615e0 con 0x560f54340b40 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560f543497d0 con 0x560f543466a0 2024-04-20T21:33:10.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f197d3fc740 10 client.0 _ll_register_callbacks cb 0x560f53fb9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f1972a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f1972a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41310/0 (socket says 172.21.15.7:41310) 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f1972a00640 1 -- 172.21.15.7:0/2212965887 learned_addr learned my addr 172.21.15.7:0/2212965887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.341+0000 7f1973e00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1973e00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54352630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 -- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 msgr2=0x560f54352630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54352630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 -- 172.21.15.7:0/2212965887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 msgr2=0x560f5434b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 -- 172.21.15.7:0/2212965887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f540846b0 con 0x560f54341530 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1973e00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54352630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1972a00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f195c011d10 tx=0x7f195c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717bdc7f2fa4765c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1961600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c01e020 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f195c0047a0 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.342+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f196c009e50 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f195c0049a0 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 10 client.15057 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560f54355a70 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f196c009e50 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f195c019070 con 0x560f54341530 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 10 client.15057 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 20 client.15057 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.343+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f54358140 0x560f543595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.344+0000 7f197d3fc740 4 client.15057 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.344+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f542b53f0 con 0x560f54358140 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.344+0000 7f1973e00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f54358140 0x560f543595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.345+0000 7f1973e00640 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f54358140 0x560f543595b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff430fbf1a7563d4 server_cookie=39322be4b4f79d2a in_seq=0 out_seq=0 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.345+0000 7f1961600640 10 client.15057 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.345+0000 7f1961600640 1 -- 172.21.15.7:0/2212965887 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f542b53f0 con 0x560f54358140 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.345+0000 7f1961600640 10 client.15057 handle_command_reply: tid=0 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.345+0000 7f197d3fc740 1 client.15057 shutdown 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 msgr2=0x560f5434b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f195c011d10 tx=0x7f195c00f790 comp rx=0 tx=0).stop 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 shutdown_connections 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f54358140 0x560f543595b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f543466a0 0x560f54352630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f54341530 0x560f5434b5f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2212965887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f1961600640 0 client.15057 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 >> 172.21.15.7:0/2212965887 conn(0x560f5433c0b0 msgr2=0x560f5433b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 shutdown_connections 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 1 -- 172.21.15.7:0/2212965887 wait complete. 2024-04-20T21:33:10.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.346+0000 7f197d3fc740 20 client.15057 trim_cache size 0 max 16384 2024-04-20T21:33:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 Processor -- start 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 -- start start 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 0x560f543525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f5433c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f543578f0 con 0x560f543466a0 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f54357fc0 con 0x560f54340b40 2024-04-20T21:33:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f197d3fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560f54358690 con 0x560f54341530 2024-04-20T21:33:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f1973e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f1972a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 0x560f543525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f1973e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54526/0 (socket says 172.21.15.7:54526) 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.367+0000 7f1973e00640 1 -- 172.21.15.7:0/2230862220 learned_addr learned my addr 172.21.15.7:0/2230862220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1973e00640 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f5433c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 -- 172.21.15.7:0/2230862220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 msgr2=0x560f5434a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 -- 172.21.15.7:0/2230862220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 msgr2=0x560f5433c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f5433c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 -- 172.21.15.7:0/2230862220 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f5433ce20 con 0x560f54341530 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1973e00640 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1972a00640 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 0x560f543525f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f195c009530 tx=0x7f195c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106cb82e61ad5b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.368+0000 7f1963e00640 1 -- 172.21.15.7:0/2230862220 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c032070 con 0x560f54341530 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f1963e00640 1 -- 172.21.15.7:0/2230862220 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195c004720 con 0x560f54341530 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f1963e00640 1 -- 172.21.15.7:0/2230862220 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c01e040 con 0x560f54341530 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 -- 172.21.15.7:0/2230862220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 msgr2=0x560f543525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 0x560f543525f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f195c009530 tx=0x7f195c009f30 comp rx=0 tx=0).stop 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 -- 172.21.15.7:0/2230862220 shutdown_connections 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f5433c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54341530 0x560f543525f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 --2- 172.21.15.7:0/2230862220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54340b40 0x560f5434a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 -- 172.21.15.7:0/2230862220 >> 172.21.15.7:0/2230862220 conn(0x560f5433c0b0 msgr2=0x560f54337280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 -- 172.21.15.7:0/2230862220 shutdown_connections 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.369+0000 7f197d3fc740 1 -- 172.21.15.7:0/2230862220 wait complete. 2024-04-20T21:33:10.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 -- start start 2024-04-20T21:33:10.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 Processor -- start 2024-04-20T21:33:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54340b40 0x560f5434ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f5434e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f54359430 con 0x560f543466a0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560f54358d60 con 0x560f54341530 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.370+0000 7f197d3fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560f54359b00 con 0x560f54340b40 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f197d3fc740 10 client.0 _ll_register_callbacks cb 0x7f195c011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41334/0 (socket says 172.21.15.7:41334) 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 -- 172.21.15.7:0/4208582355 learned_addr learned my addr 172.21.15.7:0/4208582355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1972a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f5434e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54340b40 0x560f5434ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 -- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54340b40 msgr2=0x560f5434ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54340b40 0x560f5434ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 -- 172.21.15.7:0/4208582355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 msgr2=0x560f5434e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f5434e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 -- 172.21.15.7:0/4208582355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f54351a20 con 0x560f543466a0 2024-04-20T21:33:10.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1972a00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f5434e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1973e00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc62eb8b93248f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.371+0000 7f1961600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c01e040 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c004720 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f196c02fd50 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f196c005600 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f197d3fc740 10 client.15063 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560f543be300 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f195c004720 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7f196c024040 con 0x560f543466a0 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.372+0000 7f197d3fc740 10 client.15063 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f197d3fc740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-20T21:33:10.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f197d3fc740 20 client.15063 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f543c2ca0 0x560f543c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f197d3fc740 4 client.15063 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "status"}) -- 0x560f542af260 con 0x560f543c2ca0 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f1972a00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f543c2ca0 0x560f543c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f1972a00640 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f543c2ca0 0x560f543c19b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=627a942ab142410b server_cookie=7da771d681a974a3 in_seq=0 out_seq=0 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.373+0000 7f1961600640 10 client.15063 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f1961600640 1 -- 172.21.15.7:0/4208582355 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560f542af260 con 0x560f543c2ca0 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f1961600640 10 client.15063 handle_command_reply: tid=0 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 client.15063 shutdown 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 msgr2=0x560f54351430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f196c009ca0 tx=0x7f196c02a040 comp rx=0 tx=0).stop 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 shutdown_connections 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560f543c2ca0 0x560f543c19b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560f54341530 0x560f5434e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f543466a0 0x560f54351430 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 --2- 172.21.15.7:0/4208582355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560f54340b40 0x560f5434ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f1961600640 0 client.15063 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 >> 172.21.15.7:0/4208582355 conn(0x560f5433c0b0 msgr2=0x560f5433c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 shutdown_connections 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.374+0000 7f197d3fc740 1 -- 172.21.15.7:0/4208582355 wait complete. 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f197d3fc740 20 client.15063 trim_cache size 0 max 16384 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:33:10.379 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24559, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 133.44859751000001, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 155.13448892100001 2024-04-20T21:33:10.380 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 msgr2=0x7f19580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1964034fd0 tx=0x7f19640f8530 comp rx=0 tx=0).stop 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 msgr2=0x7f197419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1964000f80 tx=0x7f19640337b0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 shutdown_connections 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1974077b60 0x7f197419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1974078500 0x7f19741a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 --2- 172.21.15.7:0/2491593066 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1974076920 0x7f197419e6b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.375+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 >> 172.21.15.7:0/2491593066 conn(0x7f19740ff570 msgr2=0x7f1974101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.376+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 shutdown_connections 2024-04-20T21:33:10.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.376+0000 7f1979000640 1 -- 172.21.15.7:0/2491593066 wait complete. 2024-04-20T21:33:10.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:10 smithi007 ceph-mon[16867]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:10.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:10 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/461447277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.390 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:10 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2514973197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 -- 172.21.15.7:0/2496812862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210102840 msgr2=0x7fd210104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 --2- 172.21.15.7:0/2496812862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210102840 0x7fd210104c30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd200009860 tx=0x7fd20002bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 -- 172.21.15.7:0/2496812862 shutdown_connections 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 --2- 172.21.15.7:0/2496812862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210107aa0 0x7fd210109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 --2- 172.21.15.7:0/2496812862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210105170 0x7fd210107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 --2- 172.21.15.7:0/2496812862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210102840 0x7fd210104c30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 -- 172.21.15.7:0/2496812862 >> 172.21.15.7:0/2496812862 conn(0x7fd2100fc250 msgr2=0x7fd2100fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 -- 172.21.15.7:0/2496812862 shutdown_connections 2024-04-20T21:33:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.603+0000 7fd214e00640 1 -- 172.21.15.7:0/2496812862 wait complete. 2024-04-20T21:33:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 Processor -- start 2024-04-20T21:33:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 -- start start 2024-04-20T21:33:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 0x7fd210198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210107aa0 0x7fd21019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd21010da70 con 0x7fd210105170 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd21010d8f0 con 0x7fd210107aa0 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.604+0000 7fd214e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd21010dbf0 con 0x7fd210102840 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41356/0 (socket says 172.21.15.7:41356) 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 -- 172.21.15.7:0/482346294 learned_addr learned my addr 172.21.15.7:0/482346294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20f400640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 0x7fd210198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20fe00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210107aa0 0x7fd21019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 -- 172.21.15.7:0/482346294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 msgr2=0x7fd210198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 0x7fd210198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 -- 172.21.15.7:0/482346294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210107aa0 msgr2=0x7fd21019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210107aa0 0x7fd21019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 -- 172.21.15.7:0/482346294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21019fd60 con 0x7fd210105170 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20f400640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 0x7fd210198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.605+0000 7fd20ea00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e9f0 tx=0x7fd1fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.606+0000 7fd207e00640 1 -- 172.21.15.7:0/482346294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00ccc0 con 0x7fd210105170 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.606+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21019fff0 con 0x7fd210105170 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.606+0000 7fd207e00640 1 -- 172.21.15.7:0/482346294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc005500 con 0x7fd210105170 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.606+0000 7fd207e00640 1 -- 172.21.15.7:0/482346294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc010690 con 0x7fd210105170 2024-04-20T21:33:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.606+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a0510 con 0x7fd210105170 2024-04-20T21:33:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.609+0000 7fd21902e740 1 Processor -- start 2024-04-20T21:33:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.609+0000 7fd21902e740 1 -- start start 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd207e00640 1 -- 172.21.15.7:0/482346294 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7fd1fc0046c0 con 0x7fd210105170 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece219f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ece21019d0 con 0x55ece21a4ab0 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ece20a8270 con 0x55ece219ef50 2024-04-20T21:33:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd21902e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ece20fd990 con 0x55ece219f940 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34504/0 (socket says 172.21.15.7:34504) 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20f400640 1 -- 172.21.15.7:0/2118621919 learned_addr learned my addr 172.21.15.7:0/2118621919 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20ea00640 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20fe00640 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece219f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20f400640 1 -- 172.21.15.7:0/2118621919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 msgr2=0x55ece219f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.610+0000 7fd20f400640 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece219f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd20f400640 1 -- 172.21.15.7:0/2118621919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 msgr2=0x55ece21a6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd20f400640 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21a6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd20f400640 1 -- 172.21.15.7:0/2118621919 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece1e142d0 con 0x55ece219f940 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd20f400640 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd200037040 tx=0x7fd200034ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f616f0df5fe7cdab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd204c00640 1 -- 172.21.15.7:0/2118621919 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200044020 con 0x55ece219f940 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd204c00640 1 -- 172.21.15.7:0/2118621919 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200034640 con 0x55ece219f940 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd204c00640 1 -- 172.21.15.7:0/2118621919 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200039040 con 0x55ece219f940 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.611+0000 7fd207e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 0x7fd1f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd207e00640 1 -- 172.21.15.7:0/482346294 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1fc0d06d0 con 0x7fd210105170 2024-04-20T21:33:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd20fe00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 0x7fd1f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 -- 172.21.15.7:0/2118621919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 msgr2=0x55ece21a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd200037040 tx=0x7fd200034ab0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 -- 172.21.15.7:0/2118621919 shutdown_connections 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21a6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a4440 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 --2- 172.21.15.7:0/2118621919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece219f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd21902e740 1 -- 172.21.15.7:0/2118621919 >> 172.21.15.7:0/2118621919 conn(0x55ece21960b0 msgr2=0x55ece21964b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.612+0000 7fd20fe00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 0x7fd1f40a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ece21a01a0 tx=0x7fd208010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.613+0000 7fd21902e740 1 -- 172.21.15.7:0/2118621919 shutdown_connections 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.613+0000 7fd21902e740 1 -- 172.21.15.7:0/2118621919 wait complete. 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.613+0000 7fd21902e740 1 -- start start 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.613+0000 7fd21902e740 1 Processor -- start 2024-04-20T21:33:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ece21a7a00 con 0x55ece21a4ab0 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ece20bb5e0 con 0x55ece219f940 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ece21a7b80 con 0x55ece219ef50 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd21902e740 10 client.0 _ll_register_callbacks cb 0x55ece1e13ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54572/0 (socket says 172.21.15.7:54572) 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 -- 172.21.15.7:0/2537944832 learned_addr learned my addr 172.21.15.7:0/2537944832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34518/0 (socket says 172.21.15.7:34518) 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20fe00640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 -- 172.21.15.7:0/2537944832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 msgr2=0x55ece21a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 -- 172.21.15.7:0/2537944832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 msgr2=0x55ece21b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20ea00640 1 -- 172.21.15.7:0/2537944832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece1ede6b0 con 0x55ece219f940 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.614+0000 7fd20f400640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd20ea00640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc098de0 tx=0x7fd1fc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7232d349076811 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc09c2e0 con 0x55ece219f940 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1fc0a5360 con 0x55ece219f940 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200034640 con 0x55ece219f940 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc0056b0 con 0x55ece219f940 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd21902e740 10 client.25181 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ece21b3e10 con 0x55ece219f940 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200034640 con 0x55ece219f940 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.615+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7fd1fc0871d0 con 0x55ece219f940 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 10 client.25181 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 10 client.25181 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 20 client.25181 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece21b6340 0x55ece21b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 4 client.25181 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ece210f3f0 con 0x55ece21b6340 2024-04-20T21:33:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd20f400640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece21b6340 0x55ece21b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd20f400640 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece21b6340 0x55ece21b77b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724f674a2cd076e3 server_cookie=2157d300145ca380 in_seq=0 out_seq=0 2024-04-20T21:33:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.616+0000 7fd1f9600640 10 client.25181 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.617+0000 7fd1f9600640 1 -- 172.21.15.7:0/2537944832 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ece210f3f0 con 0x55ece21b6340 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.617+0000 7fd1f9600640 10 client.25181 handle_command_reply: tid=0 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.617+0000 7fd21902e740 1 client.25181 shutdown 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 msgr2=0x55ece21a99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc098de0 tx=0x7fd1fc009780 comp rx=0 tx=0).stop 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 shutdown_connections 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219f940 0x55ece21a99a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece21b6340 0x55ece21b77b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece21b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 --2- 172.21.15.7:0/2537944832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece21a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd1f9600640 0 client.25181 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 >> 172.21.15.7:0/2537944832 conn(0x55ece21960b0 msgr2=0x55ece21953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 shutdown_connections 2024-04-20T21:33:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 1 -- 172.21.15.7:0/2537944832 wait complete. 2024-04-20T21:33:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.618+0000 7fd21902e740 20 client.25181 trim_cache size 0 max 16384 2024-04-20T21:33:10.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.638+0000 7fd21902e740 1 Processor -- start 2024-04-20T21:33:10.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.638+0000 7fd21902e740 1 -- start start 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece2199a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b5af0 con 0x55ece21a4ab0 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b61c0 con 0x55ece219ef50 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd21902e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b6890 con 0x55ece219f940 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece2199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54580/0 (socket says 172.21.15.7:54580) 2024-04-20T21:33:10.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 -- 172.21.15.7:0/445264036 learned_addr learned my addr 172.21.15.7:0/445264036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20ea00640 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 -- 172.21.15.7:0/445264036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 msgr2=0x55ece21a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 -- 172.21.15.7:0/445264036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 msgr2=0x55ece2199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece2199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.639+0000 7fd20f400640 1 -- 172.21.15.7:0/445264036 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece2187950 con 0x55ece219ef50 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.640+0000 7fd20f400640 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2000349e0 tx=0x7fd20002bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed25672eb0c3449 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.640+0000 7fd1fbe00640 1 -- 172.21.15.7:0/445264036 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200058070 con 0x55ece219ef50 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.640+0000 7fd1fbe00640 1 -- 172.21.15.7:0/445264036 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200034640 con 0x55ece219ef50 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.640+0000 7fd1fbe00640 1 -- 172.21.15.7:0/445264036 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200044040 con 0x55ece219ef50 2024-04-20T21:33:10.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- 172.21.15.7:0/445264036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 msgr2=0x55ece2196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2000349e0 tx=0x7fd20002bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- 172.21.15.7:0/445264036 shutdown_connections 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece21a4ab0 0x55ece2199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219f940 0x55ece21a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 --2- 172.21.15.7:0/445264036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece219ef50 0x55ece2196490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- 172.21.15.7:0/445264036 >> 172.21.15.7:0/445264036 conn(0x55ece21960b0 msgr2=0x55ece2199310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- 172.21.15.7:0/445264036 shutdown_connections 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- 172.21.15.7:0/445264036 wait complete. 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 -- start start 2024-04-20T21:33:10.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.641+0000 7fd21902e740 1 Processor -- start 2024-04-20T21:33:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece219f940 0x55ece219e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 0x55ece21b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b7630 con 0x55ece219f940 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b6f60 con 0x55ece21a4ab0 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ece21b7d00 con 0x55ece219ef50 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd21902e740 10 client.0 _ll_register_callbacks cb 0x7fd1fc004aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece219f940 0x55ece219e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 0x55ece21b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34540/0 (socket says 172.21.15.7:34540) 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 -- 172.21.15.7:0/1777230960 learned_addr learned my addr 172.21.15.7:0/1777230960 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 0x55ece21b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:54594/0 (socket says 172.21.15.7:54594) 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 -- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 msgr2=0x55ece21b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 0x55ece21b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 -- 172.21.15.7:0/1777230960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece219f940 msgr2=0x55ece219e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:10.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece219f940 0x55ece219e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.642+0000 7fd20f400640 1 -- 172.21.15.7:0/1777230960 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece219c890 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd20f400640 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd20002c0b0 tx=0x7fd20002c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b15598fc13c2de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd1f9600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200058030 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd200034640 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd200034440 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd200063050 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd21902e740 10 client.25195 fetch_fsmap learned FSMap version 23 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ece2220fa0 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.643+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd200034640 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 23) ==== 8339+0+0 (secure 0 0 0) 0x7fd200032af0 con 0x55ece219ef50 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 10 client.25195 fetch_fsmap finished waiting for FSMap version 23 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 10 client.25195 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:33:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 20 client.25195 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece22254a0 0x55ece22241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 4 client.25195 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "status"}) -- 0x55ece2109260 con 0x55ece22254a0 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.644+0000 7fd20ea00640 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece22254a0 0x55ece22241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd20ea00640 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece22254a0 0x55ece22241b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67c4c39faf312da server_cookie=141f3eaf4e1c4386 in_seq=0 out_seq=0 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd1f9600640 10 client.25195 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd1f9600640 1 -- 172.21.15.7:0/1777230960 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ece2109260 con 0x55ece22254a0 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd1f9600640 10 client.25195 handle_command_reply: tid=0 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 client.25195 shutdown 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 msgr2=0x55ece219e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd20002c0b0 tx=0x7fd20002c8c0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 shutdown_connections 2024-04-20T21:33:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ece21a4ab0 0x55ece21b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ece22254a0 0x55ece22241b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ece219f940 0x55ece219e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 --2- 172.21.15.7:0/1777230960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ece219ef50 0x55ece219e0b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd1f9600640 0 client.25195 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.645+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 >> 172.21.15.7:0/1777230960 conn(0x55ece21960b0 msgr2=0x55ece2199ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 shutdown_connections 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd21902e740 1 -- 172.21.15.7:0/1777230960 wait complete. 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd21902e740 20 client.25195 trim_cache size 0 max 16384 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:33:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "8e9da876-ff5b-11ee-bc93-c7b262605968", 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 24575, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 136.76852511999999, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 23, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 144.21153079699999 2024-04-20T21:33:10.650 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 msgr2=0x7fd1f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 0x7fd1f40a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ece21a01a0 tx=0x7fd208010040 comp rx=0 tx=0).stop 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 msgr2=0x7fd210198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.646+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e9f0 tx=0x7fd1fc00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 shutdown_connections 2024-04-20T21:33:10.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fd1f40a3c30 0x7fd1f40a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd210107aa0 0x7fd21019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd210105170 0x7fd210198680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 --2- 172.21.15.7:0/482346294 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd210102840 0x7fd210198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 >> 172.21.15.7:0/482346294 conn(0x7fd2100fc250 msgr2=0x7fd210103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 shutdown_connections 2024-04-20T21:33:10.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.647+0000 7fd214e00640 1 -- 172.21.15.7:0/482346294 wait complete. 2024-04-20T21:33:10.664 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T21:33:10.674 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T21:33:10.674 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T21:33:10.674 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T21:33:10.674 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-20T21:33:10.675 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:33:10.675 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:33:10.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-20T21:33:10.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:10 smithi016 ceph-mon[21409]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:10.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:10 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/461447277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:10 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2514973197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:10 smithi186 ceph-mon[22157]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:33:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:10 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/461447277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:10 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2514973197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 -- 172.21.15.7:0/3664226331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390077b60 msgr2=0x7f0390077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 --2- 172.21.15.7:0/3664226331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390077b60 0x7f0390077fc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0378009900 tx=0x7f037802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 -- 172.21.15.7:0/3664226331 shutdown_connections 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 --2- 172.21.15.7:0/3664226331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390077b60 0x7f0390077fc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 --2- 172.21.15.7:0/3664226331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390078500 0x7f0390078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 --2- 172.21.15.7:0/3664226331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390076920 0x7f0390076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 -- 172.21.15.7:0/3664226331 >> 172.21.15.7:0/3664226331 conn(0x7f03900ff5d0 msgr2=0x7f03901019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 -- 172.21.15.7:0/3664226331 shutdown_connections 2024-04-20T21:33:10.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.878+0000 7f038f200640 1 -- 172.21.15.7:0/3664226331 wait complete. 2024-04-20T21:33:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.879+0000 7f038f200640 1 Processor -- start 2024-04-20T21:33:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.879+0000 7f038f200640 1 -- start start 2024-04-20T21:33:10.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390076920 0x7f039019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390078500 0x7f03901a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03901140f0 con 0x7f0390078500 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0390113f70 con 0x7f0390076920 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0390114270 con 0x7f0390077b60 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f0385400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390076920 0x7f039019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f0385400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34554/0 (socket says 172.21.15.7:34554) 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f0385400640 1 -- 172.21.15.7:0/303959591 learned_addr learned my addr 172.21.15.7:0/303959591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.880+0000 7f038e800640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390078500 0x7f03901a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 -- 172.21.15.7:0/303959591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390076920 msgr2=0x7f039019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390076920 0x7f039019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 -- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390078500 msgr2=0x7f03901a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390078500 0x7f03901a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 -- 172.21.15.7:0/303959591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03901a6330 con 0x7f0390077b60 2024-04-20T21:33:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0385400640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0378005ab0 tx=0x7f037802c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378043020 con 0x7f0390077b60 2024-04-20T21:33:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03780054e0 con 0x7f0390077b60 2024-04-20T21:33:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03901a65c0 con 0x7f0390077b60 2024-04-20T21:33:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.881+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0378003ce0 con 0x7f0390077b60 2024-04-20T21:33:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.882+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03901a6b10 con 0x7f0390077b60 2024-04-20T21:33:10.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.883+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0390110990 con 0x7f0390077b60 2024-04-20T21:33:10.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.885+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) ==== 138645+0+0 (secure 0 0 0) 0x7f0378046020 con 0x7f0390077b60 2024-04-20T21:33:10.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.886+0000 7f0387400640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 0x7f03700a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.886+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f03780f2b90 con 0x7f0390077b60 2024-04-20T21:33:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.886+0000 7f038de00640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 0x7f03700a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.887+0000 7f038de00640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 0x7f03700a6050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0380005d00 tx=0x7f038000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:10.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:10.888+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f037804d020 con 0x7f0390077b60 2024-04-20T21:33:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.152+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f039002d760 con 0x7f0390077b60 2024-04-20T21:33:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.262+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f03780bb4b0 con 0x7f0390077b60 2024-04-20T21:33:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.263+0000 7f0387400640 1 -- 172.21.15.7:0/303959591 <== mon.1 v2:172.21.15.186:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f03780f4350 con 0x7f0390077b60 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 msgr2=0x7f03700a6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 0x7f03700a6050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0380005d00 tx=0x7f038000b410 comp rx=0 tx=0).stop 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 msgr2=0x7f039019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0378005ab0 tx=0x7f037802c6c0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 shutdown_connections 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f03700a3b90 0x7f03700a6050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0390078500 0x7f03901a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0390077b60 0x7f039019ec50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 --2- 172.21.15.7:0/303959591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0390076920 0x7f039019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.267+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 >> 172.21.15.7:0/303959591 conn(0x7f03900ff5d0 msgr2=0x7f03901010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.268+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 shutdown_connections 2024-04-20T21:33:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.268+0000 7f038f200640 1 -- 172.21.15.7:0/303959591 wait complete. 2024-04-20T21:33:11.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-20T21:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 -- 172.21.15.7:0/1542084911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4074940 msgr2=0x7f6ff4074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/1542084911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4074940 0x7f6ff4074da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009900 tx=0x7f6fe002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 -- 172.21.15.7:0/1542084911 shutdown_connections 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/1542084911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4074940 0x7f6ff4074da0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/1542084911 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4109e80 0x7f6ff410c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/1542084911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff4074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 -- 172.21.15.7:0/1542084911 >> 172.21.15.7:0/1542084911 conn(0x7f6ff40fd540 msgr2=0x7f6ff40ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 -- 172.21.15.7:0/1542084911 shutdown_connections 2024-04-20T21:33:11.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.482+0000 7f6ff9e00640 1 -- 172.21.15.7:0/1542084911 wait complete. 2024-04-20T21:33:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 Processor -- start 2024-04-20T21:33:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 -- start start 2024-04-20T21:33:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4109e80 0x7f6ff41a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410fe90 con 0x7f6ff4075f20 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410fd10 con 0x7f6ff4109e80 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.484+0000 7f6ff9e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4110010 con 0x7f6ff4074940 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34586/0 (socket says 172.21.15.7:34586) 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 -- 172.21.15.7:0/3161527409 learned_addr learned my addr 172.21.15.7:0/3161527409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff3400640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4109e80 0x7f6ff41a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2000640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4109e80 msgr2=0x7f6ff41a1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4109e80 0x7f6ff41a1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 msgr2=0x7f6ff419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 -- 172.21.15.7:0/3161527409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff41a2100 con 0x7f6ff4074940 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2000640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.485+0000 7f6ff2a00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400e9f0 tx=0x7f6fe400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.486+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400cd80 con 0x7f6ff4074940 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.486+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe4010910 con 0x7f6ff4074940 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.486+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff41a23f0 con 0x7f6ff4074940 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.486+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4018990 con 0x7f6ff4074940 2024-04-20T21:33:11.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.486+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff41a2960 con 0x7f6ff4074940 2024-04-20T21:33:11.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.487+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4108c40 con 0x7f6ff4074940 2024-04-20T21:33:11.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.489+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f6fe4010ac0 con 0x7f6ff4074940 2024-04-20T21:33:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.491+0000 7f6ff8a00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 0x7f6fd80a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.491+0000 7f6ff2000640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 msgr2=0x7f6fd80a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.491+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6fe40d07b0 con 0x7f6ff4074940 2024-04-20T21:33:11.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.491+0000 7f6ff2000640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 0x7f6fd80a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:11.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.494+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6fe409d160 con 0x7f6ff4074940 2024-04-20T21:33:11.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/303959591' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:11.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:11.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/303959591' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.617 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:11.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:11.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-mgr[24022]: -- 172.21.15.186:0/1785022936 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5f56965a0 con 0x55a5f566a800 2024-04-20T21:33:11.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:11.430+0000 7f6ece390200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:11.618 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:11.618+0000 7f6ece390200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:11.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/303959591' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.631 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-20T21:33:11.632 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: ignoring --setuser ceph since I am not root 2024-04-20T21:33:11.632 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: ignoring --setgroup ceph since I am not root 2024-04-20T21:33:11.632 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-mgr[22802]: -- 172.21.15.16:0/4028569740 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555832f425a0 con 0x555832f17000 2024-04-20T21:33:11.632 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:11.440+0000 7fdcf1181200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:11.632 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:11.632+0000 7fdcf1181200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.692+0000 7f6ff2000640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 msgr2=0x7f6fd80a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.692+0000 7f6ff2000640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 0x7f6fd80a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.700+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6ff402d5d0 con 0x7f6ff4074940 2024-04-20T21:33:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.706+0000 7f6ff8a00640 1 -- 172.21.15.7:0/3161527409 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7f6fe409ca60 con 0x7f6ff4074940 2024-04-20T21:33:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 msgr2=0x7f6fd80a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 0x7f6fd80a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 msgr2=0x7f6ff419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400e9f0 tx=0x7f6fe400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 shutdown_connections 2024-04-20T21:33:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f6fd80a3c70 0x7f6fd80a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ff4109e80 0x7f6ff41a1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ff4075f20 0x7f6ff419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 --2- 172.21.15.7:0/3161527409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ff4074940 0x7f6ff419a4e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 >> 172.21.15.7:0/3161527409 conn(0x7f6ff40fd540 msgr2=0x7f6ff4105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 shutdown_connections 2024-04-20T21:33:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.709+0000 7f6ff9e00640 1 -- 172.21.15.7:0/3161527409 wait complete. 2024-04-20T21:33:11.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-20T21:33:11.833 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:11.581+0000 7f178fda9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 -- 172.21.15.7:0/1092925965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 msgr2=0x7f1188074e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 --2- 172.21.15.7:0/1092925965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f1188074e40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1174009900 tx=0x7f117402bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 -- 172.21.15.7:0/1092925965 shutdown_connections 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 --2- 172.21.15.7:0/1092925965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f118810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 --2- 172.21.15.7:0/1092925965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f1188074e40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 --2- 172.21.15.7:0/1092925965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 0x7f11880743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 -- 172.21.15.7:0/1092925965 >> 172.21.15.7:0/1092925965 conn(0x7f11880fd390 msgr2=0x7f11880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 -- 172.21.15.7:0/1092925965 shutdown_connections 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.916+0000 7f1187e00640 1 -- 172.21.15.7:0/1092925965 wait complete. 2024-04-20T21:33:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 Processor -- start 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 -- start start 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 0x7f118819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f11881a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f118810fcb0 con 0x7f1188075fa0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f118810fb30 con 0x7f1188109ce0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1187e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f118810fe30 con 0x7f11880749c0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1186a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.918+0000 7f1186000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 0x7f118819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1186a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34604/0 (socket says 172.21.15.7:34604) 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1186a00640 1 -- 172.21.15.7:0/3023510872 learned_addr learned my addr 172.21.15.7:0/3023510872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f11881a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 msgr2=0x7f118819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 msgr2=0x7f118819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 0x7f118819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 -- 172.21.15.7:0/3023510872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11881a1f10 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1186a00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f1187400640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f11881a18a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fd0 tx=0x7f118000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.919+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118000ce20 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.920+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1180005020 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.920+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11881a2200 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.920+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180010720 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.920+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11881a2720 con 0x7f1188109ce0 2024-04-20T21:33:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.921+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1188108a10 con 0x7f1188109ce0 2024-04-20T21:33:11.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.923+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7f1180005540 con 0x7f1188109ce0 2024-04-20T21:33:11.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.924+0000 7f117f400640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 0x7f116c0a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:11.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.924+0000 7f1186a00640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 msgr2=0x7f116c0a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.924+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f11800d0ab0 con 0x7f1188109ce0 2024-04-20T21:33:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.924+0000 7f1186a00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 0x7f116c0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:11.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:11.928+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f118009d4c0 con 0x7f1188109ce0 2024-04-20T21:33:11.944 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:11 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:11.739+0000 7fdcf1181200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:11.965 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:11 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:11.722+0000 7f6ece390200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:12.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.125+0000 7f1186a00640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 msgr2=0x7f116c0a6120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:12.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.125+0000 7f1186a00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 0x7f116c0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.138+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f1188062b60 con 0x7f1188109ce0 2024-04-20T21:33:12.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.144+0000 7f117f400640 1 -- 172.21.15.7:0/3023510872 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7f118009cdc0 con 0x7f1188109ce0 2024-04-20T21:33:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.146+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 msgr2=0x7f116c0a6120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.146+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 0x7f116c0a6120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 msgr2=0x7f11881a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:12.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f11881a18a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1180009fd0 tx=0x7f118000ef10 comp rx=0 tx=0).stop 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 shutdown_connections 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f116c0a3c60 0x7f116c0a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188109ce0 0x7f11881a18a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1188075fa0 0x7f118819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 --2- 172.21.15.7:0/3023510872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f11880749c0 0x7f118819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:12.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 >> 172.21.15.7:0/3023510872 conn(0x7f11880fd390 msgr2=0x7f1188105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:12.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 shutdown_connections 2024-04-20T21:33:12.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:12.147+0000 7f1187e00640 1 -- 172.21.15.7:0/3023510872 wait complete. 2024-04-20T21:33:12.162 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:11.833+0000 7f178fda9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:12.163 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:11 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:11.982+0000 7f178fda9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:33:12.166 INFO:teuthology.run_tasks:Running task kclient... 2024-04-20T21:33:12.193 INFO:tasks.kclient:Mounting kernel clients... 2024-04-20T21:33:12.194 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-20T21:33:12.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:12.194 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-20T21:33:12.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:12.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-20T21:33:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-04-20T21:33:12.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:33:12.332 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-20T21:33:12.332 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:33:12.332 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:12.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-20T21:33:12.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:12.060+0000 7fdcf1181200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:12.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:12.148+0000 7fdcf1181200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-mon[21409]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-mon[21409]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-20T21:33:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:12 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:12.035+0000 7f6ece390200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:12.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:12.121+0000 7f6ece390200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-mon[22157]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-mon[22157]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-20T21:33:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:12 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.528 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:12.409+0000 7f178fda9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:33:12.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-mon[16867]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:33:12.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-20T21:33:12.528 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-mon[16867]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-20T21:33:12.529 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.529 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' 2024-04-20T21:33:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:33:12.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:12.528+0000 7f178fda9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:33:13.668 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:13.322+0000 7f6ece390200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:13.668 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:13.494+0000 7f6ece390200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:13.668 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:13.668+0000 7f6ece390200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:13.694 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:13.394+0000 7fdcf1181200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:13.694 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:13.571+0000 7fdcf1181200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:14.045 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:13 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:13.747+0000 7fdcf1181200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:14.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:13 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:13.960+0000 7f6ece390200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:14.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:14 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:14.055+0000 7f6ece390200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:14.350 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:14.045+0000 7fdcf1181200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:14.350 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:14.141+0000 7fdcf1181200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:14.401 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:14 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:14.151+0000 7f178fda9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:33:14.656 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:14 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:14.412+0000 7f178fda9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:14.694 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:14.350+0000 7fdcf1181200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:14.694 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:14.550+0000 7fdcf1181200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:14.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:14 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:14.260+0000 7f6ece390200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:14.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:14 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:14.457+0000 7f6ece390200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 -- 172.21.15.7:0/1125732064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e810cf70 msgr2=0x7fc7e810f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 --2- 172.21.15.7:0/1125732064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e810cf70 0x7fc7e810f430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000b0a0 tx=0x7fc7e000b3b0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 -- 172.21.15.7:0/1125732064 shutdown_connections 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 --2- 172.21.15.7:0/1125732064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e810cf70 0x7fc7e810f430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 --2- 172.21.15.7:0/1125732064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e8103b40 0x7fc7e8103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 --2- 172.21.15.7:0/1125732064 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 0x7fc7e810c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 -- 172.21.15.7:0/1125732064 >> 172.21.15.7:0/1125732064 conn(0x7fc7e80fd370 msgr2=0x7fc7e80ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.940+0000 7fc7eca00700 1 -- 172.21.15.7:0/1125732064 shutdown_connections 2024-04-20T21:33:14.941 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:14 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:14.663+0000 7f178fda9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:33:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.941+0000 7fc7eca00700 1 -- 172.21.15.7:0/1125732064 wait complete. 2024-04-20T21:33:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.942+0000 7fc7eca00700 1 Processor -- start 2024-04-20T21:33:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 -- start start 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 0x7fc7e8101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e810cf70 0x7fc7e8102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112d00 con 0x7fc7e810cf70 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112b80 con 0x7fc7e8103b40 2024-04-20T21:33:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7eca00700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112e80 con 0x7fc7e8104460 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7e6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7e6800700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47516/0 (socket says 172.21.15.7:47516) 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7e6800700 1 -- 172.21.15.7:0/2412809261 learned_addr learned my addr 172.21.15.7:0/2412809261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.943+0000 7fc7e5e00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 0x7fc7e8101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e7200700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e810cf70 0x7fc7e8102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 msgr2=0x7fc7e8101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 0x7fc7e8101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e810cf70 msgr2=0x7fc7e8102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e810cf70 0x7fc7e8102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 -- 172.21.15.7:0/2412809261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a76e0 con 0x7fc7e8103b40 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7e6800700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e9a0 tx=0x7fc7d800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.944+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800ca10 con 0x7fc7e8103b40 2024-04-20T21:33:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.945+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d800cbb0 con 0x7fc7e8103b40 2024-04-20T21:33:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.945+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e81a79d0 con 0x7fc7e8103b40 2024-04-20T21:33:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.945+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8018680 con 0x7fc7e8103b40 2024-04-20T21:33:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.945+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e81a7f60 con 0x7fc7e8103b40 2024-04-20T21:33:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.946+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e8069740 con 0x7fc7e8103b40 2024-04-20T21:33:14.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.948+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138662+0+0 (secure 0 0 0) 0x7fc7d8010430 con 0x7fc7e8103b40 2024-04-20T21:33:14.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.950+0000 7fc7df400700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 0x7fc7cc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.950+0000 7fc7e5e00700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 msgr2=0x7fc7cc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.950+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc7d80d05f0 con 0x7fc7e8103b40 2024-04-20T21:33:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.950+0000 7fc7e5e00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 0x7fc7cc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:14.953+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc7d809cfa0 con 0x7fc7e8103b40 2024-04-20T21:33:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.151+0000 7fc7e5e00700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 msgr2=0x7fc7cc0a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.151+0000 7fc7e5e00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 0x7fc7cc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:15.201 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:15 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:15.072+0000 7f178fda9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:33:15.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.229+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc7e81a8130 con 0x7fc7e8103b40 2024-04-20T21:33:15.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.229+0000 7fc7df400700 1 -- 172.21.15.7:0/2412809261 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fc7d809c8a0 con 0x7fc7e8103b40 2024-04-20T21:33:15.229 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 msgr2=0x7fc7cc0a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 0x7fc7cc0a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 msgr2=0x7fc7e81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800e9a0 tx=0x7fc7d800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 shutdown_connections 2024-04-20T21:33:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7e810cf70 0x7fc7e8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7e8103b40 0x7fc7e81017a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7fc7cc0a3d10 0x7fc7cc0a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 --2- 172.21.15.7:0/2412809261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc7e8104460 0x7fc7e8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 >> 172.21.15.7:0/2412809261 conn(0x7fc7e80fd370 msgr2=0x7fc7e810f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 shutdown_connections 2024-04-20T21:33:15.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:15.232+0000 7fc7eca00700 1 -- 172.21.15.7:0/2412809261 wait complete. 2024-04-20T21:33:15.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:15 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2412809261' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:15.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:15.176+0000 7fdcf1181200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:15.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:15 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:15.068+0000 7f6ece390200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:15.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:15 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2412809261' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:15.474 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:15 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2412809261' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-20T21:33:15.474 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:15 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:15.201+0000 7f178fda9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:33:15.750 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:15 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:15.474+0000 7f178fda9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:33:15.794 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:15 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:15.496+0000 7f6ece390200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:15.795 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:15 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:15.593+0000 7f6ece390200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:15.795 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:15 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:15.795+0000 7f6ece390200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:15.923 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:15.615+0000 7fdcf1181200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:15.924 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:15.714+0000 7fdcf1181200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:15.924 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:15.923+0000 7fdcf1181200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:15.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:33:15.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:33:15.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:33:15.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:15.985 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-20T21:33:16.027 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:33:16.028 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41921sec preferred_lft 41921sec 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:33:16.029 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-20T21:33:16.030 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:33:16.030 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:16.030 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-20T21:33:16.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:33:16.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-04-20T21:33:16.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-04-20T21:33:16.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:33:16.031 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-20T21:33:16.032 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:15 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:15.750+0000 7f178fda9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:33:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:16.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:16.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:16.498 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:33:16.567 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-20T21:33:16.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:16.571 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-20T21:33:16.617 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-20T21:33:16.618 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:33:16.618 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T21:33:16.618 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T21:33:16.618 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T21:33:16.619 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-20T21:33:16.667 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:16 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:16.327+0000 7f6ece390200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:16.667 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:16 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:16.421+0000 7f6ece390200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:16.756 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:16 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:16.650+0000 7f178fda9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:33:16.821 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:16.473+0000 7fdcf1181200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:16.822 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:16.569+0000 7fdcf1181200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:16.965 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:16 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:16.668+0000 7f6ece390200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:16.965 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:16 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:16.802+0000 7f6ece390200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:16.965 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:16 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:16.899+0000 7f6ece390200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:17.181 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:16.821+0000 7fdcf1181200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:17.181 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:16 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:16.963+0000 7fdcf1181200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:17.181 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:17 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:17.061+0000 7fdcf1181200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:17.181 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:17 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:17.181+0000 7fdcf1181200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:17.323 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:17 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:17.020+0000 7f6ece390200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:17.323 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:17 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:17.324+0000 7f6ece390200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:17.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:17.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:17.334 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-20T21:33:17.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:17.382 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-04-20T21:33:17.475 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:17.475 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-20T21:33:17.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:33:17.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T21:33:17.475 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-20T21:33:17.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:17 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:17.277+0000 7f178fda9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:33:17.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:17 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:17.415+0000 7f178fda9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:33:17.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:17.657 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T21:33:17.657 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T21:33:17.658 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-20T21:33:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:17.864 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:17 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:17.712+0000 7f178fda9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:33:17.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:17.944 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:33:17.944 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-20T21:33:17.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-20T21:33:17.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:33:17.945 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-20T21:33:17.945 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:17 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:17.492+0000 7fdcf1181200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:18.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:18.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:18.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:33:18.140 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:18.187 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:33:18.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:33:18.188 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:18.215 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:17 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:17.930+0000 7f6ece390200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:18.216 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z[24018]: 2024-04-20T21:33:18.147+0000 7f6ece390200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:18.271 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-20T21:33:18.271 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-20T21:33:18.271 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6d643e23-dc51-46e4-949f-950c298bfaa4 2024-04-20T21:33:18.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:18.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6d643e23-dc51-46e4-949f-950c298bfaa4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T21:33:18.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:18.122+0000 7fdcf1181200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:18.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y[22798]: 2024-04-20T21:33:18.350+0000 7fdcf1181200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: Standby manager daemon z restarted 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: Standby manager daemon z started 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:18.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:18 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: Standby manager daemon z restarted 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: Standby manager daemon z started 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:18.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:18.544 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T21:33:18.545 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T21:33:18.545 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-04-20T21:33:18.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: Standby manager daemon z restarted 2024-04-20T21:33:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: Standby manager daemon z started 2024-04-20T21:33:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-20T21:33:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-20T21:33:18.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:18 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.186:0/221898767' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:18.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:18.470+0000 7f178fda9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:33:18.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:18.601+0000 7f178fda9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:33:19.170 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:18 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:18.981+0000 7f178fda9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:33:19.170 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:19.169+0000 7f178fda9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:33:19.293 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-20T21:33:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-20T21:33:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:invalid new device string format 2024-04-20T21:33:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.16:3300,172.21.15.186:3300" 2024-04-20T21:33:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.7:3300,172.21.15.16:3300,172.21.15.186:3300:/volumes/qa/sv_1/6d643e23-dc51-46e4-949f-950c298bfaa4 2024-04-20T21:33:19.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=8e9da876-ff5b-11ee-bc93-c7b262605968" will pass to kernel 2024-04-20T21:33:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-20T21:33:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-20T21:33:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-20T21:33:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-20T21:33:19.297 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-20T21:33:19.301 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-20T21:33:19.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:19.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:19.383 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:19.472 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:19.310+0000 7f178fda9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: Standby manager daemon y restarted 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: Standby manager daemon y started 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:19.473 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-mon[16867]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:19.503 INFO:teuthology.orchestra.run.smithi007.stdout:client15120 2024-04-20T21:33:19.503 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:33:19.504 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:33:19.554 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:19.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: Standby manager daemon y restarted 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: Standby manager daemon y started 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:19.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:19 smithi016 ceph-mon[21409]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-20T21:33:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: Standby manager daemon y restarted 2024-04-20T21:33:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: Standby manager daemon y started 2024-04-20T21:33:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-20T21:33:19.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-20T21:33:19.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-20T21:33:19.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:19 smithi186 ceph-mon[22157]: from='mgr.? 172.21.15.16:0/2910785128' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-20T21:33:19.718 INFO:teuthology.orchestra.run.smithi007.stdout:client15120 2024-04-20T21:33:19.718 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-20T21:33:19.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:19.472+0000 7f178fda9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:33:19.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T21:33:19.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:19.832 INFO:teuthology.orchestra.run.smithi007.stdout:client15120 2024-04-20T21:33:19.832 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968.client15120/status') 2024-04-20T21:33:19.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:33:19.970 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.15120 (3)192.168.0.1:0/2105009159 2024-04-20T21:33:19.971 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-04-20T21:33:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-04-20T21:33:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-04-20T21:33:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:70 bytes copied, 6.0201e-05 s, 1.2 MB/s 2024-04-20T21:33:19.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:33:19.984 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T21:33:19.994 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:33:19.994 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:33:19.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.180+0000 7f7fd1600640 1 -- 172.21.15.7:0/2085589664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109eb0 msgr2=0x7f7fcc10c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.180+0000 7f7fd1600640 1 --2- 172.21.15.7:0/2085589664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc10c340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009ca0 tx=0x7f7fc402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 -- 172.21.15.7:0/2085589664 shutdown_connections 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 --2- 172.21.15.7:0/2085589664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc10c340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 --2- 172.21.15.7:0/2085589664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc074e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 --2- 172.21.15.7:0/2085589664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc0743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 -- 172.21.15.7:0/2085589664 >> 172.21.15.7:0/2085589664 conn(0x7f7fcc0fd580 msgr2=0x7f7fcc0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 -- 172.21.15.7:0/2085589664 shutdown_connections 2024-04-20T21:33:20.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.181+0000 7f7fd1600640 1 -- 172.21.15.7:0/2085589664 wait complete. 2024-04-20T21:33:20.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 Processor -- start 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 -- start start 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10fe70 con 0x7f7fcc075fa0 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10fcf0 con 0x7f7fcc109eb0 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.182+0000 7f7fd1600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc10fff0 con 0x7f7fcc0749c0 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcaa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33328/0 (socket says 172.21.15.7:33328) 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fca000640 1 -- 172.21.15.7:0/1024989374 learned_addr learned my addr 172.21.15.7:0/1024989374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:20.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 msgr2=0x7f7fcc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 msgr2=0x7f7fcc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcb400640 1 -- 172.21.15.7:0/1024989374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc1a2080 con 0x7f7fcc109eb0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fcaa00640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.183+0000 7f7fca000640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fcb400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc1a1a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fc402c630 tx=0x7f7fc4032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4041020 con 0x7f7fcc109eb0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc1a2310 con 0x7f7fcc109eb0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4034cd0 con 0x7f7fcc109eb0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4039040 con 0x7f7fcc109eb0 2024-04-20T21:33:20.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.184+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a27d0 con 0x7f7fcc109eb0 2024-04-20T21:33:20.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.185+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc108be0 con 0x7f7fcc109eb0 2024-04-20T21:33:20.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.187+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 138662+0+0 (secure 0 0 0) 0x7f7fc4044020 con 0x7f7fcc109eb0 2024-04-20T21:33:20.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.189+0000 7f7fc3400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:20.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.189+0000 7f7fcaa00640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 msgr2=0x7f7fb00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:20.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.189+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7fc40f21d0 con 0x7f7fcc109eb0 2024-04-20T21:33:20.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.189+0000 7f7fcaa00640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:33:20.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.193+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7fc404b020 con 0x7f7fcc109eb0 2024-04-20T21:33:20.242 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:19 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:19.863+0000 7f178fda9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:33:20.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.390+0000 7f7fcaa00640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 msgr2=0x7f7fb00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:20.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.390+0000 7f7fcaa00640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T21:33:20.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.438+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7fcc062b60 con 0x7f7fcc109eb0 2024-04-20T21:33:20.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:20 smithi007 ceph-mon[16867]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T21:33:20.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:20 smithi016 ceph-mon[21409]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T21:33:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:20 smithi186 ceph-mon[22157]: mgrmap e21: x(active, since 10m), standbys: y, z 2024-04-20T21:33:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.791+0000 7f7fcaa00640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 msgr2=0x7f7fb00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.7:6800/2375060083 2024-04-20T21:33:20.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:20.791+0000 7f7fcaa00640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-20T21:33:20.983 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:20.679+0000 7f178fda9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:33:21.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.010+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f7fc4003970 con 0x7f7fcc109eb0 2024-04-20T21:33:21.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.011+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 msgr2=0x7f7fb00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:33:21.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.011+0000 7f7fc3400640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.245 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:20.982+0000 7f178fda9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:33:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.261+0000 7f7fc3400640 1 -- 172.21.15.7:0/1024989374 <== mon.2 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f7fc40bec10 con 0x7f7fcc109eb0 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 msgr2=0x7f7fcc1a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc1a1a10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7fc402c630 tx=0x7f7fc4032f20 comp rx=0 tx=0).stop 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 shutdown_connections 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:6800/2375060083,v1:172.21.15.7:6801/2375060083] conn(0x7f7fb00a3bf0 0x7f7fb00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fcc109eb0 0x7f7fcc1a1a10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fcc075fa0 0x7f7fcc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 --2- 172.21.15.7:0/1024989374 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fcc0749c0 0x7f7fcc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 >> 172.21.15.7:0/1024989374 conn(0x7f7fcc0fd580 msgr2=0x7f7fcc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.264+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 shutdown_connections 2024-04-20T21:33:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.265+0000 7f7fd1600640 1 -- 172.21.15.7:0/1024989374 wait complete. 2024-04-20T21:33:21.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-20T21:33:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 -- 172.21.15.7:0/1573936566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f41041f0 msgr2=0x7f87f4104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 --2- 172.21.15.7:0/1573936566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f41041f0 0x7f87f4104610 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009900 tx=0x7f87e402bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 -- 172.21.15.7:0/1573936566 shutdown_connections 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 --2- 172.21.15.7:0/1573936566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f4105da0 0x7f87f410c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 --2- 172.21.15.7:0/1573936566 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f4105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 --2- 172.21.15.7:0/1573936566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f41041f0 0x7f87f4104610 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 -- 172.21.15.7:0/1573936566 >> 172.21.15.7:0/1573936566 conn(0x7f87f40ff780 msgr2=0x7f87f4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.490+0000 7f87f9400640 1 -- 172.21.15.7:0/1573936566 shutdown_connections 2024-04-20T21:33:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.491+0000 7f87f9400640 1 -- 172.21.15.7:0/1573936566 wait complete. 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.491+0000 7f87f9400640 1 Processor -- start 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 -- start start 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 0x7f87f41a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410fed0 con 0x7f87f4105da0 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f87f410fd50 con 0x7f87f41041f0 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f9400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f87f4110050 con 0x7f87f41053e0 2024-04-20T21:33:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47682/0 (socket says 172.21.15.7:47682) 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f2800640 1 -- 172.21.15.7:0/620510184 learned_addr learned my addr 172.21.15.7:0/620510184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.492+0000 7f87f3200640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 0x7f87f41a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 -- 172.21.15.7:0/620510184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 msgr2=0x7f87f419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 -- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 msgr2=0x7f87f41a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 0x7f87f41a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 -- 172.21.15.7:0/620510184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f41a20f0 con 0x7f87f41041f0 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f1e00640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f3200640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 0x7f87f41a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f2800640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87e4000c00 tx=0x7f87e402c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4043020 con 0x7f87f41041f0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f41a2380 con 0x7f87f41041f0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87e40054e0 con 0x7f87f41041f0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.493+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003d70 con 0x7f87f41041f0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.494+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f41a28a0 con 0x7f87f41041f0 2024-04-20T21:33:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.495+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f4050540 con 0x7f87f41041f0 2024-04-20T21:33:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.498+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f87e4046020 con 0x7f87f41041f0 2024-04-20T21:33:21.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.500+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f87e40f21b0 con 0x7f87f41041f0 2024-04-20T21:33:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:21.501+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f87e404d020 con 0x7f87f41041f0 2024-04-20T21:33:21.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.387+0000 7f16e5800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.387+0000 7f16e5800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.387+0000 7f16e5800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.387+0000 7f16e5800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.387+0000 7f16e5800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.388+0000 7f16e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.388+0000 7f16e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.388+0000 7f16e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.388+0000 7f16e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.533 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:21.388+0000 7f16e0800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1024989374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: Active manager daemon x restarted 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: Activating manager daemon x 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: mgrmap e22: x(active, starting, since 0.0146124s), standbys: y, z 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:21.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:21.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:21.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: Manager daemon x is now available 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:21.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:21 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1024989374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: Active manager daemon x restarted 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: Activating manager daemon x 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: mgrmap e22: x(active, starting, since 0.0146124s), standbys: y, z 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:21.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:21.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: Manager daemon x is now available 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:21.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:21 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1024989374' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: Active manager daemon x restarted 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: Activating manager daemon x 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: osdmap e89: 12 total, 12 up, 12 in 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: mgrmap e22: x(active, starting, since 0.0146124s), standbys: y, z 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-20T21:33:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-20T21:33:21.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-20T21:33:21.718 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: Manager daemon x is now available 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:21.719 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:21 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-20T21:33:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.019+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f87e40ba510 con 0x7f87f41041f0 2024-04-20T21:33:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.020+0000 7f87eb400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 0x7f87d80acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.020+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f87e4001d30 con 0x7f87d80aa820 2024-04-20T21:33:22.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.027+0000 7f87f1e00640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 0x7f87d80acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.032+0000 7f87f1e00640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 0x7f87d80acc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f419b9e0 tx=0x7f87e000b390 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.060+0000 7f87eb400640 1 -- 172.21.15.7:0/620510184 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f87e4001d30 con 0x7f87d80aa820 2024-04-20T21:33:22.061 INFO:teuthology.orchestra.run.smithi007.stdout:{} 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 msgr2=0x7f87d80acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 0x7f87d80acc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87f419b9e0 tx=0x7f87e000b390 comp rx=0 tx=0).stop 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 msgr2=0x7f87f419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87e4000c00 tx=0x7f87e402c6c0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 shutdown_connections 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87f4105da0 0x7f87f41a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f87f41053e0 0x7f87f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f87d80aa820 0x7f87d80acc10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 --2- 172.21.15.7:0/620510184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87f41041f0 0x7f87f419a4d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 >> 172.21.15.7:0/620510184 conn(0x7f87f40ff780 msgr2=0x7f87f41086e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 shutdown_connections 2024-04-20T21:33:22.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.063+0000 7f87f9400640 1 -- 172.21.15.7:0/620510184 wait complete. 2024-04-20T21:33:22.079 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-20T21:33:22.115 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-20T21:33:22.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 -- 172.21.15.7:0/4077764146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 msgr2=0x7fd49810c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 --2- 172.21.15.7:0/4077764146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd49810c500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd490009be0 tx=0x7fd49002bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 -- 172.21.15.7:0/4077764146 shutdown_connections 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 --2- 172.21.15.7:0/4077764146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd49810c500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 --2- 172.21.15.7:0/4077764146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 0x7fd498105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 --2- 172.21.15.7:0/4077764146 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 0x7fd4981044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 -- 172.21.15.7:0/4077764146 >> 172.21.15.7:0/4077764146 conn(0x7fd4980ff630 msgr2=0x7fd498101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:22.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 -- 172.21.15.7:0/4077764146 shutdown_connections 2024-04-20T21:33:22.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.361+0000 7fd49ee00640 1 -- 172.21.15.7:0/4077764146 wait complete. 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 Processor -- start 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 -- start start 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 0x7fd49819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 0x7fd49819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810fda0 con 0x7fd4981052b0 2024-04-20T21:33:22.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810fc20 con 0x7fd498105c70 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.362+0000 7fd49ee00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd49810ff20 con 0x7fd4981040c0 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47712/0 (socket says 172.21.15.7:47712) 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 -- 172.21.15.7:0/658033329 learned_addr learned my addr 172.21.15.7:0/658033329 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49da00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 0x7fd49819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49d000640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 0x7fd49819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 -- 172.21.15.7:0/658033329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 msgr2=0x7fd49819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 0x7fd49819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 -- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 msgr2=0x7fd49819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 0x7fd49819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 -- 172.21.15.7:0/658033329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4981a1f10 con 0x7fd498105c70 2024-04-20T21:33:22.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.363+0000 7fd49e400640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd490036040 tx=0x7fd4900337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:22.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.366+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490043020 con 0x7fd498105c70 2024-04-20T21:33:22.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.366+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49002ca60 con 0x7fd498105c70 2024-04-20T21:33:22.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.366+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490034690 con 0x7fd498105c70 2024-04-20T21:33:22.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.366+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4981a21a0 con 0x7fd498105c70 2024-04-20T21:33:22.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.366+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981a2740 con 0x7fd498105c70 2024-04-20T21:33:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.368+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49810b310 con 0x7fd498105c70 2024-04-20T21:33:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.369+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fd490046020 con 0x7fd498105c70 2024-04-20T21:33:22.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.370+0000 7fd496000640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 0x7fd4800a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.371+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd4900f2750 con 0x7fd498105c70 2024-04-20T21:33:22.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.371+0000 7fd49da00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 0x7fd4800a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.371+0000 7fd49da00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 0x7fd4800a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009530 tx=0x7fd48c009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.375+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd49004d030 con 0x7fd498105c70 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-mon[16867]: mgrmap e23: x(active, since 1.03891s), standbys: y, z 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:22.048+0000 7f16dea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:22.534 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:22.048+0000 7f16dea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:22.535 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:22.048+0000 7f16dea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:22.535 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:22.048+0000 7f16dea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:22.535 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:33:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:33:22.048+0000 7f16dea00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:33:22.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.580+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd498062b60 con 0x7fd498105c70 2024-04-20T21:33:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.582+0000 7fd496000640 1 -- 172.21.15.7:0/658033329 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22859 (secure 0 0 0) 0x7fd4900bf310 con 0x7fd498105c70 2024-04-20T21:33:22.582 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:22.583 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:33:20.988355+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 msgr2=0x7fd4800a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 0x7fd4800a5ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009530 tx=0x7fd48c009260 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 msgr2=0x7fd4981a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd490036040 tx=0x7fd4900337b0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 shutdown_connections 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4800a3a10 0x7fd4800a5ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd498105c70 0x7fd4981a18a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4981052b0 0x7fd49819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 --2- 172.21.15.7:0/658033329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4981040c0 0x7fd49819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 >> 172.21.15.7:0/658033329 conn(0x7fd4980ff630 msgr2=0x7fd4981084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 shutdown_connections 2024-04-20T21:33:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.585+0000 7fd49ee00640 1 -- 172.21.15.7:0/658033329 wait complete. 2024-04-20T21:33:22.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:22 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T21:33:22.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:22 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:33:22.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:22 smithi186 ceph-mon[22157]: mgrmap e23: x(active, since 1.03891s), standbys: y, z 2024-04-20T21:33:22.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:22 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.590 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:22 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:22.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:22 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-20T21:33:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:22 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:33:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:22 smithi016 ceph-mon[21409]: mgrmap e23: x(active, since 1.03891s), standbys: y, z 2024-04-20T21:33:22.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:22 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:22 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 -- 172.21.15.7:0/693403789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 msgr2=0x7fbaf01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/693403789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf01023c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009860 tx=0x7fbae002bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 -- 172.21.15.7:0/693403789 shutdown_connections 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/693403789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103b50 0x7fbaf010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/693403789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0103190 0x7fbaf0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/693403789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf01023c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 -- 172.21.15.7:0/693403789 >> 172.21.15.7:0/693403789 conn(0x7fbaf00fd4f0 msgr2=0x7fbaf00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:22.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 -- 172.21.15.7:0/693403789 shutdown_connections 2024-04-20T21:33:22.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.804+0000 7fbaf4e00640 1 -- 172.21.15.7:0/693403789 wait complete. 2024-04-20T21:33:22.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 Processor -- start 2024-04-20T21:33:22.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 -- start start 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 0x7fbaf01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0103b50 0x7fbaf019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010dc30 con 0x7fbaf0103b50 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010dab0 con 0x7fbaf0103190 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.806+0000 7fbaf4e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010ddb0 con 0x7fbaf0101fa0 2024-04-20T21:33:22.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:36310/0 (socket says 172.21.15.7:36310) 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 -- 172.21.15.7:0/3956843023 learned_addr learned my addr 172.21.15.7:0/3956843023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaed400640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 0x7fbaf01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 -- 172.21.15.7:0/3956843023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 msgr2=0x7fbaf01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 0x7fbaf01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 -- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0103b50 msgr2=0x7fbaf019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0103b50 0x7fbaf019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 -- 172.21.15.7:0/3956843023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf019feb0 con 0x7fbaf0101fa0 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaed400640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 0x7fbaf01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:22.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.807+0000 7fbaede00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbae00349e0 tx=0x7fbae0032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.808+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0041020 con 0x7fbaf0101fa0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.808+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbae002c9f0 con 0x7fbaf0101fa0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.808+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0039040 con 0x7fbaf0101fa0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.808+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf01a0140 con 0x7fbaf0101fa0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.808+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf01a05e0 con 0x7fbaf0101fa0 2024-04-20T21:33:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.809+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf0109390 con 0x7fbaf0101fa0 2024-04-20T21:33:22.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.811+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fbae0044020 con 0x7fbaf0101fa0 2024-04-20T21:33:22.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.813+0000 7fbae6a00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 0x7fbad40a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:22.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.813+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbae00f19e0 con 0x7fbaf0101fa0 2024-04-20T21:33:22.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.813+0000 7fbaed400640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 0x7fbad40a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:22.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.814+0000 7fbaed400640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 0x7fbad40a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbaf01997a0 tx=0x7fbadc009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:22.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:22.817+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbae004b020 con 0x7fbaf0101fa0 2024-04-20T21:33:23.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.030+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbaf0062c00 con 0x7fbaf0101fa0 2024-04-20T21:33:23.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.031+0000 7fbae6a00640 1 -- 172.21.15.7:0/3956843023 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fbae00be5a0 con 0x7fbaf0101fa0 2024-04-20T21:33:23.031 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 msgr2=0x7fbad40a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 0x7fbad40a5ef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbaf01997a0 tx=0x7fbadc009260 comp rx=0 tx=0).stop 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 msgr2=0x7fbaf0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbae00349e0 tx=0x7fbae0032430 comp rx=0 tx=0).stop 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 shutdown_connections 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbad40a3a30 0x7fbad40a5ef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf0103190 0x7fbaf01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaf0103b50 0x7fbaf019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 --2- 172.21.15.7:0/3956843023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbaf0101fa0 0x7fbaf0198290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 >> 172.21.15.7:0/3956843023 conn(0x7fbaf00fd4f0 msgr2=0x7fbaf0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 shutdown_connections 2024-04-20T21:33:23.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.034+0000 7fbaf4e00640 1 -- 172.21.15.7:0/3956843023 wait complete. 2024-04-20T21:33:23.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 -- 172.21.15.7:0/3401405969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 msgr2=0x7f2c34104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 --2- 172.21.15.7:0/3401405969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34104c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009900 tx=0x7f2c2402bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 -- 172.21.15.7:0/3401405969 shutdown_connections 2024-04-20T21:33:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 --2- 172.21.15.7:0/3401405969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 --2- 172.21.15.7:0/3401405969 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c34109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 --2- 172.21.15.7:0/3401405969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34104c40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 -- 172.21.15.7:0/3401405969 >> 172.21.15.7:0/3401405969 conn(0x7f2c340fc240 msgr2=0x7f2c340fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 -- 172.21.15.7:0/3401405969 shutdown_connections 2024-04-20T21:33:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.251+0000 7f2c33e00640 1 -- 172.21.15.7:0/3401405969 wait complete. 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.252+0000 7f2c33e00640 1 Processor -- start 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 -- start start 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3410daa0 con 0x7f2c34102850 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3410d920 con 0x7f2c34105180 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3410dc20 con 0x7f2c34107ab0 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c32a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:56836/0 (socket says 172.21.15.7:56836) 2024-04-20T21:33:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.253+0000 7f2c33400640 1 -- 172.21.15.7:0/923168732 learned_addr learned my addr 172.21.15.7:0/923168732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 -- 172.21.15.7:0/923168732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 msgr2=0x7f2c3419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 -- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 msgr2=0x7f2c34198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 -- 172.21.15.7:0/923168732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3419fd40 con 0x7f2c34105180 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c33400640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c32000640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34198660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e9c0 tx=0x7f2c2000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000ccc0 con 0x7f2c34105180 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.254+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c20010910 con 0x7f2c34105180 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.255+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3419ffd0 con 0x7f2c34105180 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.255+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c200188c0 con 0x7f2c34105180 2024-04-20T21:33:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.255+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c341a04f0 con 0x7f2c34105180 2024-04-20T21:33:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.256+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c340fdd30 con 0x7f2c34105180 2024-04-20T21:33:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.258+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f2c20010ac0 con 0x7f2c34105180 2024-04-20T21:33:23.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.260+0000 7f2c2b400640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 0x7f2c180a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.260+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2c200d0430 con 0x7f2c34105180 2024-04-20T21:33:23.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.260+0000 7f2c32a00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 0x7f2c180a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.261+0000 7f2c32a00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 0x7f2c180a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c240 tx=0x7f2c2402df90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.263+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f2c2009c9a0 con 0x7f2c34105180 2024-04-20T21:33:23.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.392+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7f2c20099010 con 0x7f2c34105180 2024-04-20T21:33:23.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.483+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c34062b60 con 0x7f2c34105180 2024-04-20T21:33:23.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.484+0000 7f2c2b400640 1 -- 172.21.15.7:0/923168732 <== mon.2 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f2c200570f0 con 0x7f2c34105180 2024-04-20T21:33:23.484 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:33:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 msgr2=0x7f2c180a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 0x7f2c180a5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c240 tx=0x7f2c2402df90 comp rx=0 tx=0).stop 2024-04-20T21:33:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 msgr2=0x7f2c34198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34198660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000e9c0 tx=0x7f2c2000ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 shutdown_connections 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2c180a39b0 0x7f2c180a5e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c34105180 0x7f2c34198660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c34107ab0 0x7f2c3419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.486+0000 7f2c33e00640 1 --2- 172.21.15.7:0/923168732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c34102850 0x7f2c34198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.487+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 >> 172.21.15.7:0/923168732 conn(0x7f2c340fc240 msgr2=0x7f2c34103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.487+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 shutdown_connections 2024-04-20T21:33:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.487+0000 7f2c33e00640 1 -- 172.21.15.7:0/923168732 wait complete. 2024-04-20T21:33:23.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: [20/Apr/2024:21:33:21] ENGINE Bus STARTING 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: [20/Apr/2024:21:33:21] ENGINE Serving on https://172.21.15.7:7150 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: [20/Apr/2024:21:33:21] ENGINE Serving on http://172.21.15.7:8765 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: [20/Apr/2024:21:33:21] ENGINE Bus STARTED 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.641 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/658033329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:23.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3956843023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:23.642 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:23 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: [20/Apr/2024:21:33:21] ENGINE Bus STARTING 2024-04-20T21:33:23.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: [20/Apr/2024:21:33:21] ENGINE Serving on https://172.21.15.7:7150 2024-04-20T21:33:23.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: [20/Apr/2024:21:33:21] ENGINE Serving on http://172.21.15.7:8765 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: [20/Apr/2024:21:33:21] ENGINE Bus STARTED 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/658033329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3956843023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:23.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:23 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: [20/Apr/2024:21:33:21] ENGINE Bus STARTING 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: [20/Apr/2024:21:33:21] ENGINE Serving on https://172.21.15.7:7150 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: [20/Apr/2024:21:33:21] ENGINE Serving on http://172.21.15.7:8765 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: [20/Apr/2024:21:33:21] ENGINE Bus STARTED 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='client.25274 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:23.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:23.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/658033329' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:23.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3956843023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:23.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:23 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 -- 172.21.15.7:0/2513912701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18103980 msgr2=0x7faa1810a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2513912701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18103980 0x7faa1810a210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faa10009be0 tx=0x7faa1002bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 -- 172.21.15.7:0/2513912701 shutdown_connections 2024-04-20T21:33:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2513912701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18103980 0x7faa1810a210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2513912701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18102fc0 0x7faa18103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2513912701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18101dd0 0x7faa181021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.722+0000 7faa1ec00640 1 -- 172.21.15.7:0/2513912701 >> 172.21.15.7:0/2513912701 conn(0x7faa180fd340 msgr2=0x7faa180ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.723+0000 7faa1ec00640 1 -- 172.21.15.7:0/2513912701 shutdown_connections 2024-04-20T21:33:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.723+0000 7faa1ec00640 1 -- 172.21.15.7:0/2513912701 wait complete. 2024-04-20T21:33:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 Processor -- start 2024-04-20T21:33:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 -- start start 2024-04-20T21:33:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18102fc0 0x7faa1819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18103980 0x7faa181a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa1810da80 con 0x7faa18103980 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa1810d900 con 0x7faa18101dd0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ec00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faa1810dc00 con 0x7faa18102fc0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57060/0 (socket says 172.21.15.7:57060) 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1d800640 1 -- 172.21.15.7:0/2453446945 learned_addr learned my addr 172.21.15.7:0/2453446945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1ce00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18102fc0 0x7faa1819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.724+0000 7faa1e200640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18103980 0x7faa181a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 -- 172.21.15.7:0/2453446945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18102fc0 msgr2=0x7faa1819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18102fc0 0x7faa1819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 -- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18103980 msgr2=0x7faa181a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18103980 0x7faa181a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 -- 172.21.15.7:0/2453446945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa181a1e80 con 0x7faa18101dd0 2024-04-20T21:33:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1d800640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9f0 tx=0x7faa0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ccc0 con 0x7faa18101dd0 2024-04-20T21:33:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0c0035f0 con 0x7faa18101dd0 2024-04-20T21:33:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa181a2170 con 0x7faa18101dd0 2024-04-20T21:33:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.725+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ce60 con 0x7faa18101dd0 2024-04-20T21:33:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.726+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a2710 con 0x7faa18101dd0 2024-04-20T21:33:23.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.727+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa181a29d0 con 0x7faa18101dd0 2024-04-20T21:33:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.729+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7faa0c0037a0 con 0x7faa18101dd0 2024-04-20T21:33:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.731+0000 7faa16000640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 0x7faa000a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.731+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7faa0c0d0c40 con 0x7faa18101dd0 2024-04-20T21:33:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.731+0000 7faa1ce00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 0x7faa000a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.732+0000 7faa1ce00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 0x7faa000a62e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa1819b770 tx=0x7faa04005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.733+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7faa0c09d160 con 0x7faa18101dd0 2024-04-20T21:33:23.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.938+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faa181a2ba0 con 0x7faa18101dd0 2024-04-20T21:33:23.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.939+0000 7faa16000640 1 -- 172.21.15.7:0/2453446945 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7faa0c09ca60 con 0x7faa18101dd0 2024-04-20T21:33:23.938 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:33:23.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 msgr2=0x7faa000a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 0x7faa000a62e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faa1819b770 tx=0x7faa04005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 msgr2=0x7faa1819a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9f0 tx=0x7faa0c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 shutdown_connections 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa18103980 0x7faa181a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faa18102fc0 0x7faa1819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faa000a3e20 0x7faa000a62e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 --2- 172.21.15.7:0/2453446945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18101dd0 0x7faa1819a260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 >> 172.21.15.7:0/2453446945 conn(0x7faa180fd340 msgr2=0x7faa18074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 shutdown_connections 2024-04-20T21:33:23.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:23.941+0000 7faa1ec00640 1 -- 172.21.15.7:0/2453446945 wait complete. 2024-04-20T21:33:23.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:24.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 -- 172.21.15.7:0/1751275847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec10a060 msgr2=0x7f7eec1108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/1751275847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec10a060 0x7f7eec1108f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009ca0 tx=0x7f7ee402c100 comp rx=0 tx=0).stop 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 -- 172.21.15.7:0/1751275847 shutdown_connections 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/1751275847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec10a060 0x7f7eec1108f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/1751275847 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec1096a0 0x7f7eec109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/1751275847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec108500 0x7f7eec108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 -- 172.21.15.7:0/1751275847 >> 172.21.15.7:0/1751275847 conn(0x7f7eec077450 msgr2=0x7f7eec079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 -- 172.21.15.7:0/1751275847 shutdown_connections 2024-04-20T21:33:24.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.153+0000 7f7ef1a00640 1 -- 172.21.15.7:0/1751275847 wait complete. 2024-04-20T21:33:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 Processor -- start 2024-04-20T21:33:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 -- start start 2024-04-20T21:33:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 0x7f7eec19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec10a060 0x7f7eec1a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec114140 con 0x7f7eec108500 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec113fc0 con 0x7f7eec1096a0 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7ef1a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec1142c0 con 0x7f7eec10a060 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7eeb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 0x7f7eec19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7eeb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57072/0 (socket says 172.21.15.7:57072) 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.154+0000 7f7eeb400640 1 -- 172.21.15.7:0/3112110762 learned_addr learned my addr 172.21.15.7:0/3112110762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7ef1000640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec10a060 0x7f7eec1a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 -- 172.21.15.7:0/3112110762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec10a060 msgr2=0x7f7eec1a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec10a060 0x7f7eec1a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 -- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 msgr2=0x7f7eec19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 0x7f7eec19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 -- 172.21.15.7:0/3112110762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec1a6330 con 0x7f7eec1096a0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eebe00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 0x7f7eec19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7eeb400640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e9f0 tx=0x7f7ed400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400cd80 con 0x7f7eec1096a0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed4005500 con 0x7f7eec1096a0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4010690 con 0x7f7eec1096a0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.155+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec1a6620 con 0x7f7eec1096a0 2024-04-20T21:33:24.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.156+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec1a6b70 con 0x7f7eec1096a0 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.157+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec10f770 con 0x7f7eec1096a0 2024-04-20T21:33:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.159+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7f7ed40046c0 con 0x7f7eec1096a0 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.160+0000 7f7ee8c00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 0x7f7ed00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.160+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7ed40d0f20 con 0x7f7eec1096a0 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.160+0000 7f7eebe00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 0x7f7ed00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.161+0000 7f7eebe00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 0x7f7ed00a62e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7eec109500 tx=0x7f7edc005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.162+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7ed4053ff0 con 0x7f7eec1096a0 2024-04-20T21:33:24.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.370+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7eec02d820 con 0x7f7eec1096a0 2024-04-20T21:33:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.378+0000 7f7ee8c00640 1 -- 172.21.15.7:0/3112110762 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22859 (secure 0 0 0) 0x7f7ed409d440 con 0x7f7eec1096a0 2024-04-20T21:33:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:24.377 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:33:20.988355+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 msgr2=0x7f7ed00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 0x7f7ed00a62e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7eec109500 tx=0x7f7edc005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 msgr2=0x7f7eec19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400e9f0 tx=0x7f7ed400edf0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 shutdown_connections 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7ed00a3e20 0x7f7ed00a62e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eec1096a0 0x7f7eec19ec50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7eec10a060 0x7f7eec1a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 --2- 172.21.15.7:0/3112110762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7eec108500 0x7f7eec19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 >> 172.21.15.7:0/3112110762 conn(0x7f7eec077450 msgr2=0x7f7eec10c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 shutdown_connections 2024-04-20T21:33:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.380+0000 7f7ef1a00640 1 -- 172.21.15.7:0/3112110762 wait complete. 2024-04-20T21:33:24.398 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:24.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 -- 172.21.15.7:0/1286120549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02401041b0 msgr2=0x7f02401045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 --2- 172.21.15.7:0/1286120549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02401041b0 0x7f02401045d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0230009900 tx=0x7f023002bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 -- 172.21.15.7:0/1286120549 shutdown_connections 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 --2- 172.21.15.7:0/1286120549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240105d60 0x7f024010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 --2- 172.21.15.7:0/1286120549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401053a0 0x7f0240105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 --2- 172.21.15.7:0/1286120549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02401041b0 0x7f02401045d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 -- 172.21.15.7:0/1286120549 >> 172.21.15.7:0/1286120549 conn(0x7f02400ff720 msgr2=0x7f0240101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 -- 172.21.15.7:0/1286120549 shutdown_connections 2024-04-20T21:33:24.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.609+0000 7f0246800640 1 -- 172.21.15.7:0/1286120549 wait complete. 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.610+0000 7f0246800640 1 Processor -- start 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 -- start start 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 0x7f024019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 0x7f02401a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f024010fe90 con 0x7f02401053a0 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f024010fd10 con 0x7f0240105d60 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f0246800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0240110010 con 0x7f02401041b0 2024-04-20T21:33:24.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 0x7f024019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:56902/0 (socket says 172.21.15.7:56902) 2024-04-20T21:33:24.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 0x7f024019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57340/0 (socket says 172.21.15.7:57340) 2024-04-20T21:33:24.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023ea00640 1 -- 172.21.15.7:0/882639616 learned_addr learned my addr 172.21.15.7:0/882639616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:24.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.611+0000 7f023fe00640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 0x7f02401a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 -- 172.21.15.7:0/882639616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 msgr2=0x7f024019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 -- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 msgr2=0x7f024019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 0x7f024019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 -- 172.21.15.7:0/882639616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02401a20d0 con 0x7f0240105d60 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023f400640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.612+0000 7f023fe00640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 0x7f02401a1a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0238009fd0 tx=0x7f023800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.613+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800ce20 con 0x7f0240105d60 2024-04-20T21:33:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.613+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0238005020 con 0x7f0240105d60 2024-04-20T21:33:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.613+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238010720 con 0x7f0240105d60 2024-04-20T21:33:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.613+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02401a23c0 con 0x7f0240105d60 2024-04-20T21:33:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.613+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02401a2910 con 0x7f0240105d60 2024-04-20T21:33:24.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.614+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f024010b550 con 0x7f0240105d60 2024-04-20T21:33:24.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.619+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7f0238005540 con 0x7f0240105d60 2024-04-20T21:33:24.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.619+0000 7f0237e00640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 0x7f02240a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:24.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.619+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f02380d1590 con 0x7f0240105d60 2024-04-20T21:33:24.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.619+0000 7f023f400640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 0x7f02240a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:24.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.620+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f023809db10 con 0x7f0240105d60 2024-04-20T21:33:24.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.621+0000 7f023f400640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 0x7f02240a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f023002c170 tx=0x7f023002df90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T21:33:24.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/923168732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2453446945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:24 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T21:33:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/923168732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2453446945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:24 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-20T21:33:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/923168732' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2453446945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:24.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:24 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:24.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.834+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f024002d5d0 con 0x7f0240105d60 2024-04-20T21:33:24.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.835+0000 7f0237e00640 1 -- 172.21.15.7:0/882639616 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f023809d410 con 0x7f0240105d60 2024-04-20T21:33:24.835 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 msgr2=0x7f02240a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 0x7f02240a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f023002c170 tx=0x7f023002df90 comp rx=0 tx=0).stop 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 msgr2=0x7f02401a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 0x7f02401a1a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0238009fd0 tx=0x7f023800ef10 comp rx=0 tx=0).stop 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 shutdown_connections 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02240a3e30 0x7f02240a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0240105d60 0x7f02401a1a60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02401053a0 0x7f024019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 --2- 172.21.15.7:0/882639616 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f02401041b0 0x7f024019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 >> 172.21.15.7:0/882639616 conn(0x7f02400ff720 msgr2=0x7f0240108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 shutdown_connections 2024-04-20T21:33:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:24.838+0000 7f0246800640 1 -- 172.21.15.7:0/882639616 wait complete. 2024-04-20T21:33:24.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 -- 172.21.15.7:0/456879041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 msgr2=0x7fd3701045e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 --2- 172.21.15.7:0/456879041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd3701045e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd364009900 tx=0x7fd36402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 -- 172.21.15.7:0/456879041 shutdown_connections 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 --2- 172.21.15.7:0/456879041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd370105d70 0x7fd37010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 --2- 172.21.15.7:0/456879041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3701053b0 0x7fd370105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 --2- 172.21.15.7:0/456879041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd3701045e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 -- 172.21.15.7:0/456879041 >> 172.21.15.7:0/456879041 conn(0x7fd3700ff750 msgr2=0x7fd370101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 -- 172.21.15.7:0/456879041 shutdown_connections 2024-04-20T21:33:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.054+0000 7fd377a00640 1 -- 172.21.15.7:0/456879041 wait complete. 2024-04-20T21:33:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 Processor -- start 2024-04-20T21:33:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 -- start start 2024-04-20T21:33:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3701053b0 0x7fd37019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 0x7fd3701a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd37010fea0 con 0x7fd370105d70 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd37010fd20 con 0x7fd3701041c0 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd377a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd370110020 con 0x7fd3701053b0 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd375000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd375000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57114/0 (socket says 172.21.15.7:57114) 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.055+0000 7fd375000640 1 -- 172.21.15.7:0/943940879 learned_addr learned my addr 172.21.15.7:0/943940879 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 0x7fd3701a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd36fe00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3701053b0 0x7fd37019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 -- 172.21.15.7:0/943940879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3701053b0 msgr2=0x7fd37019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3701053b0 0x7fd37019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 -- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 msgr2=0x7fd3701a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 0x7fd3701a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 -- 172.21.15.7:0/943940879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3701a2110 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 0x7fd3701a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd375000640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd36402efd0 tx=0x7fd364005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.056+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364041070 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.057+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3640044d0 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.057+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3701a23a0 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.057+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364034440 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.057+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701a28b0 con 0x7fd3701041c0 2024-04-20T21:33:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.058+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd370069260 con 0x7fd3701041c0 2024-04-20T21:33:25.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.060+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7fd364044070 con 0x7fd3701041c0 2024-04-20T21:33:25.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.061+0000 7fd36d600640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 0x7fd3580a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.061+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd3640f23d0 con 0x7fd3701041c0 2024-04-20T21:33:25.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.061+0000 7fd36fe00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 0x7fd3580a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.062+0000 7fd36fe00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 0x7fd3580a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0095f0 tx=0x7fd35c005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.065+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd3640bea00 con 0x7fd3701041c0 2024-04-20T21:33:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.297+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd370062b60 con 0x7fd3701041c0 2024-04-20T21:33:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.299+0000 7fd36d600640 1 -- 172.21.15.7:0/943940879 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fd3640be300 con 0x7fd3701041c0 2024-04-20T21:33:25.297 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:33:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.300+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 msgr2=0x7fd3580a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.300+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 0x7fd3580a62f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0095f0 tx=0x7fd35c005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.300+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 msgr2=0x7fd37019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.300+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd36402efd0 tx=0x7fd364005d50 comp rx=0 tx=0).stop 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 shutdown_connections 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd370105d70 0x7fd3701a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3701053b0 0x7fd37019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd3580a3e30 0x7fd3580a62f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 --2- 172.21.15.7:0/943940879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3701041c0 0x7fd37019a4f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 >> 172.21.15.7:0/943940879 conn(0x7fd3700ff750 msgr2=0x7fd3701086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 shutdown_connections 2024-04-20T21:33:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.301+0000 7fd377a00640 1 -- 172.21.15.7:0/943940879 wait complete. 2024-04-20T21:33:25.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2743763704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 msgr2=0x7fdc08105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2743763704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc08105820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009860 tx=0x7fdbf402bb70 comp rx=0 tx=0).stop 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2743763704 shutdown_connections 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2743763704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc08105820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2743763704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc08105d60 0x7fdc0810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2743763704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc081041b0 0x7fdc081045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2743763704 >> 172.21.15.7:0/2743763704 conn(0x7fdc080ff6e0 msgr2=0x7fdc08101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2743763704 shutdown_connections 2024-04-20T21:33:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.518+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2743763704 wait complete. 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.519+0000 7fdc0fc00640 1 Processor -- start 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.519+0000 7fdc0fc00640 1 -- start start 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc081041b0 0x7fdc0819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc08105d60 0x7fdc081a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0810fe40 con 0x7fdc08105d60 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0810fcc0 con 0x7fdc081053a0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0fc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0810ffc0 con 0x7fdc081041b0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc081041b0 0x7fdc0819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57132/0 (socket says 172.21.15.7:57132) 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 -- 172.21.15.7:0/2312296872 learned_addr learned my addr 172.21.15.7:0/2312296872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc0dc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc08105d60 0x7fdc081a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 -- 172.21.15.7:0/2312296872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc081041b0 msgr2=0x7fdc0819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc081041b0 0x7fdc0819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 -- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc08105d60 msgr2=0x7fdc081a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc08105d60 0x7fdc081a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.520+0000 7fdc07e00640 1 -- 172.21.15.7:0/2312296872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc081a2050 con 0x7fdc081053a0 2024-04-20T21:33:25.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc07e00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40349e0 tx=0x7fdbf4032450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4041030 con 0x7fdc081053a0 2024-04-20T21:33:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc081a22e0 con 0x7fdc081053a0 2024-04-20T21:33:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf402c510 con 0x7fdc081053a0 2024-04-20T21:33:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4039040 con 0x7fdc081053a0 2024-04-20T21:33:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.521+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc081a27d0 con 0x7fdc081053a0 2024-04-20T21:33:25.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.522+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0810b4d0 con 0x7fdc081053a0 2024-04-20T21:33:25.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.524+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138816+0+0 (secure 0 0 0) 0x7fdbf4044070 con 0x7fdc081053a0 2024-04-20T21:33:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.526+0000 7fdc05600640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 0x7fdbf00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.526+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdbf40f1d80 con 0x7fdc081053a0 2024-04-20T21:33:25.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.526+0000 7fdc0d200640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 0x7fdbf00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.527+0000 7fdc0d200640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 0x7fdbf00a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009530 tx=0x7fdbfc009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.530+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdbf40be3b0 con 0x7fdc081053a0 2024-04-20T21:33:25.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.547+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fdbf40ba320 con 0x7fdc081053a0 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3112110762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/882639616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/943940879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:25 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3112110762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:25.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/882639616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/943940879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:25 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.747+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdc0802d820 con 0x7fdc081053a0 2024-04-20T21:33:25.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.748+0000 7fdc05600640 1 -- 172.21.15.7:0/2312296872 <== mon.2 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fdbf40ad6d0 con 0x7fdc081053a0 2024-04-20T21:33:25.747 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 msgr2=0x7fdbf00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 0x7fdbf00a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009530 tx=0x7fdbfc009260 comp rx=0 tx=0).stop 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 msgr2=0x7fdc0819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40349e0 tx=0x7fdbf4032450 comp rx=0 tx=0).stop 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 shutdown_connections 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbf00a3db0 0x7fdbf00a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc081053a0 0x7fdc0819a970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc08105d60 0x7fdc081a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 --2- 172.21.15.7:0/2312296872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc081041b0 0x7fdc0819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 >> 172.21.15.7:0/2312296872 conn(0x7fdc080ff6e0 msgr2=0x7fdc081086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 shutdown_connections 2024-04-20T21:33:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.751+0000 7fdc0fc00640 1 -- 172.21.15.7:0/2312296872 wait complete. 2024-04-20T21:33:25.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3112110762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:25.767 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/882639616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.768 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/943940879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.769 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:25 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:25.770 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-20T21:33:25.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 -- 172.21.15.7:0/1948819650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 msgr2=0x7f83a010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 --2- 172.21.15.7:0/1948819650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a010a1d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8398009ca0 tx=0x7f839802c100 comp rx=0 tx=0).stop 2024-04-20T21:33:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 -- 172.21.15.7:0/1948819650 shutdown_connections 2024-04-20T21:33:25.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 --2- 172.21.15.7:0/1948819650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a010a1d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 --2- 172.21.15.7:0/1948819650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0105450 0x7f83a0107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 --2- 172.21.15.7:0/1948819650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0102b20 0x7f83a0104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 -- 172.21.15.7:0/1948819650 >> 172.21.15.7:0/1948819650 conn(0x7f83a00fc530 msgr2=0x7f83a00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.972+0000 7f83a4e00640 1 -- 172.21.15.7:0/1948819650 shutdown_connections 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.973+0000 7f83a4e00640 1 -- 172.21.15.7:0/1948819650 wait complete. 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 Processor -- start 2024-04-20T21:33:25.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 -- start start 2024-04-20T21:33:25.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0102b20 0x7f83a0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a0101010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010dc50 con 0x7f83a0107d80 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010dad0 con 0x7f83a0105450 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.974+0000 7f83a4e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83a010ddd0 con 0x7f83a0102b20 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a0101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0102b20 0x7f83a0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57154/0 (socket says 172.21.15.7:57154) 2024-04-20T21:33:25.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 -- 172.21.15.7:0/486626983 learned_addr learned my addr 172.21.15.7:0/486626983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 -- 172.21.15.7:0/486626983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0102b20 msgr2=0x7f83a0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0102b20 0x7f83a0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 -- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 msgr2=0x7f83a0101010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a0101010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:25.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.975+0000 7f839d400640 1 -- 172.21.15.7:0/486626983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a01018a0 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f839d400640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e9f0 tx=0x7f838c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c00ccc0 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f838c005500 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c010690 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a01a2360 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.976+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a01a2860 con 0x7f83a0105450 2024-04-20T21:33:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.978+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00fe020 con 0x7f83a0105450 2024-04-20T21:33:25.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.979+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f838c0046c0 con 0x7f83a0105450 2024-04-20T21:33:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.981+0000 7f8396a00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 0x7f83840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.981+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f838c0d0e80 con 0x7f83a0105450 2024-04-20T21:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.981+0000 7f839de00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 0x7f83840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.982+0000 7f839de00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 0x7f83840a6300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83900095f0 tx=0x7f8390005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:25.985+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f838c09d2c0 con 0x7f83a0105450 2024-04-20T21:33:26.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.243+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a002d780 con 0x7f83a0105450 2024-04-20T21:33:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.244+0000 7f8396a00640 1 -- 172.21.15.7:0/486626983 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f838c09cbc0 con 0x7f83a0105450 2024-04-20T21:33:26.243 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:26.243 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:33:20.441301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:33:26.244 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-04-20T21:33:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.247+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 msgr2=0x7f83840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.247+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 0x7f83840a6300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f83900095f0 tx=0x7f8390005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.247+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 msgr2=0x7f83a0100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.247+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e9f0 tx=0x7f838c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 shutdown_connections 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83840a3e40 0x7f83840a6300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0105450 0x7f83a0100ad0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83a0107d80 0x7f83a0101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 --2- 172.21.15.7:0/486626983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83a0102b20 0x7f83a0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 >> 172.21.15.7:0/486626983 conn(0x7f83a00fc530 msgr2=0x7f83a0103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 shutdown_connections 2024-04-20T21:33:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.248+0000 7f83a4e00640 1 -- 172.21.15.7:0/486626983 wait complete. 2024-04-20T21:33:26.266 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-20T21:33:26.266 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-20T21:33:26.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 -- 172.21.15.7:0/2115481678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103a00 msgr2=0x7f5b4010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 --2- 172.21.15.7:0/2115481678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103a00 0x7f5b4010a320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009ca0 tx=0x7f5b3c02c100 comp rx=0 tx=0).stop 2024-04-20T21:33:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 -- 172.21.15.7:0/2115481678 shutdown_connections 2024-04-20T21:33:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 --2- 172.21.15.7:0/2115481678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103a00 0x7f5b4010a320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 --2- 172.21.15.7:0/2115481678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103040 0x7f5b401034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 --2- 172.21.15.7:0/2115481678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 -- 172.21.15.7:0/2115481678 >> 172.21.15.7:0/2115481678 conn(0x7f5b400fd420 msgr2=0x7f5b400ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 -- 172.21.15.7:0/2115481678 shutdown_connections 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.465+0000 7f5b47200640 1 -- 172.21.15.7:0/2115481678 wait complete. 2024-04-20T21:33:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.467+0000 7f5b47200640 1 Processor -- start 2024-04-20T21:33:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.467+0000 7f5b47200640 1 -- start start 2024-04-20T21:33:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103040 0x7f5b40198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103a00 0x7f5b4019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010db50 con 0x7f5b40103040 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010d9d0 con 0x7f5b40101e60 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b47200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4010dcd0 con 0x7f5b40103a00 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103040 0x7f5b40198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57166/0 (socket says 172.21.15.7:57166) 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45e00640 1 -- 172.21.15.7:0/2940436835 learned_addr learned my addr 172.21.15.7:0/2940436835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b46800640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103a00 0x7f5b4019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45e00640 1 -- 172.21.15.7:0/2940436835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103a00 msgr2=0x7f5b4019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.468+0000 7f5b45e00640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103a00 0x7f5b4019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b45e00640 1 -- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103040 msgr2=0x7f5b40198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b45e00640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103040 0x7f5b40198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b45e00640 1 -- 172.21.15.7:0/2940436835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019fd20 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b45e00640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9c0 tx=0x7f5b3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400cde0 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34010910 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4019ffb0 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34018a10 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.469+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401a0470 con 0x7f5b40101e60 2024-04-20T21:33:26.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.471+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b401091a0 con 0x7f5b40101e60 2024-04-20T21:33:26.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.472+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f5b3401f020 con 0x7f5b40101e60 2024-04-20T21:33:26.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.474+0000 7f5b3aa00640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 0x7f5b280a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.474+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5b340d0c60 con 0x7f5b40101e60 2024-04-20T21:33:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.474+0000 7f5b45400640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 0x7f5b280a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.475+0000 7f5b45400640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 0x7f5b280a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b40199610 tx=0x7f5b30005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.478+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5b3409d0a0 con 0x7f5b40101e60 2024-04-20T21:33:26.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2312296872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:26.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:26 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/486626983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:33:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2312296872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:26.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:26 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/486626983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:26.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.704+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b400623a0 con 0x7f5b280a3e30 2024-04-20T21:33:26.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.716+0000 7f5b3aa00640 1 -- 172.21.15.7:0/2940436835 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257974 (secure 0 0 0) 0x7f5b400623a0 con 0x7f5b280a3e30 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-20T21:33:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2312296872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:26.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:26 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/486626983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 msgr2=0x7f5b280a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 0x7f5b280a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5b40199610 tx=0x7f5b30005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 msgr2=0x7f5b40198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e9c0 tx=0x7f5b3400ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 shutdown_connections 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b40103a00 0x7f5b4019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b40103040 0x7f5b40198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5b280a3e30 0x7f5b280a62f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 --2- 172.21.15.7:0/2940436835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40101e60 0x7f5b40198100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 >> 172.21.15.7:0/2940436835 conn(0x7f5b400fd420 msgr2=0x7f5b40106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 shutdown_connections 2024-04-20T21:33:26.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.721+0000 7f5b47200640 1 -- 172.21.15.7:0/2940436835 wait complete. 2024-04-20T21:33:26.740 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-20T21:33:25.026155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188103,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1129,"num_read_kb":726,"num_write":478,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352708,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589116,"statfs":{"total":1151940427776,"available":1151579254784,"internally_reserved":0,"allocated":12128256,"data_stored":8216769,"data_compressed":165237,"data_compressed_allocated":212992,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.975290"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179119+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:33:21.179119+0000","last_peered":"2024-04-20T21:33:21.179119+0000","last_clean":"2024-04-20T21:33:21.179119+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:33:21.179119+0000","last_undegraded":"2024-04-20T21:33:21.179119+0000","last_fullsized":"2024-04-20T21:33:21.179119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210790+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:33:21.210790+0000","last_peered":"2024-04-20T21:33:21.210790+0000","last_clean":"2024-04-20T21:33:21.210790+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:33:21.210790+0000","last_undegraded":"2024-04-20T21:33:21.210790+0000","last_fullsized":"2024-04-20T21:33:21.210790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210576+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:33:21.210576+0000","last_peered":"2024-04-20T21:33:21.210576+0000","last_clean":"2024-04-20T21:33:21.210576+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:33:21.210576+0000","last_undegraded":"2024-04-20T21:33:21.210576+0000","last_fullsized":"2024-04-20T21:33:21.210576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290607+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:33:21.290607+0000","last_peered":"2024-04-20T21:33:21.290607+0000","last_clean":"2024-04-20T21:33:21.290607+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:33:21.290607+0000","last_undegraded":"2024-04-20T21:33:21.290607+0000","last_fullsized":"2024-04-20T21:33:21.290607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208826+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:33:21.208826+0000","last_peered":"2024-04-20T21:33:21.208826+0000","last_clean":"2024-04-20T21:33:21.208826+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:33:21.208826+0000","last_undegraded":"2024-04-20T21:33:21.208826+0000","last_fullsized":"2024-04-20T21:33:21.208826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000919+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:33:21.000919+0000","last_peered":"2024-04-20T21:33:21.000919+0000","last_clean":"2024-04-20T21:33:21.000919+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:33:21.000919+0000","last_undegraded":"2024-04-20T21:33:21.000919+0000","last_fullsized":"2024-04-20T21:33:21.000919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178892+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:33:21.178892+0000","last_peered":"2024-04-20T21:33:21.178892+0000","last_clean":"2024-04-20T21:33:21.178892+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:33:21.178892+0000","last_undegraded":"2024-04-20T21:33:21.178892+0000","last_fullsized":"2024-04-20T21:33:21.178892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208900+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:33:21.208900+0000","last_peered":"2024-04-20T21:33:21.208900+0000","last_clean":"2024-04-20T21:33:21.208900+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:33:21.208900+0000","last_undegraded":"2024-04-20T21:33:21.208900+0000","last_fullsized":"2024-04-20T21:33:21.208900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191648+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:33:21.191648+0000","last_peered":"2024-04-20T21:33:21.191648+0000","last_clean":"2024-04-20T21:33:21.191648+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:33:21.191648+0000","last_undegraded":"2024-04-20T21:33:21.191648+0000","last_fullsized":"2024-04-20T21:33:21.191648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001738+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:33:21.001738+0000","last_peered":"2024-04-20T21:33:21.001738+0000","last_clean":"2024-04-20T21:33:21.001738+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:33:21.001738+0000","last_undegraded":"2024-04-20T21:33:21.001738+0000","last_fullsized":"2024-04-20T21:33:21.001738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290577+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:33:21.290577+0000","last_peered":"2024-04-20T21:33:21.290577+0000","last_clean":"2024-04-20T21:33:21.290577+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:33:21.290577+0000","last_undegraded":"2024-04-20T21:33:21.290577+0000","last_fullsized":"2024-04-20T21:33:21.290577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209784+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:33:21.209784+0000","last_peered":"2024-04-20T21:33:21.209784+0000","last_clean":"2024-04-20T21:33:21.209784+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:33:21.209784+0000","last_undegraded":"2024-04-20T21:33:21.209784+0000","last_fullsized":"2024-04-20T21:33:21.209784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210061+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:33:21.210061+0000","last_peered":"2024-04-20T21:33:21.210061+0000","last_clean":"2024-04-20T21:33:21.210061+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:33:21.210061+0000","last_undegraded":"2024-04-20T21:33:21.210061+0000","last_fullsized":"2024-04-20T21:33:21.210061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210055+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:33:21.210055+0000","last_peered":"2024-04-20T21:33:21.210055+0000","last_clean":"2024-04-20T21:33:21.210055+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:33:21.210055+0000","last_undegraded":"2024-04-20T21:33:21.210055+0000","last_fullsized":"2024-04-20T21:33:21.210055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208278+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:33:21.208278+0000","last_peered":"2024-04-20T21:33:21.208278+0000","last_clean":"2024-04-20T21:33:21.208278+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:33:21.208278+0000","last_undegraded":"2024-04-20T21:33:21.208278+0000","last_fullsized":"2024-04-20T21:33:21.208278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179317+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:33:21.179317+0000","last_peered":"2024-04-20T21:33:21.179317+0000","last_clean":"2024-04-20T21:33:21.179317+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:33:21.179317+0000","last_undegraded":"2024-04-20T21:33:21.179317+0000","last_fullsized":"2024-04-20T21:33:21.179317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002145+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:33:21.002145+0000","last_peered":"2024-04-20T21:33:21.002145+0000","last_clean":"2024-04-20T21:33:21.002145+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:33:21.002145+0000","last_undegraded":"2024-04-20T21:33:21.002145+0000","last_fullsized":"2024-04-20T21:33:21.002145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003416+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:33:21.003416+0000","last_peered":"2024-04-20T21:33:21.003416+0000","last_clean":"2024-04-20T21:33:21.003416+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:33:21.003416+0000","last_undegraded":"2024-04-20T21:33:21.003416+0000","last_fullsized":"2024-04-20T21:33:21.003416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001834+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:33:21.001834+0000","last_peered":"2024-04-20T21:33:21.001834+0000","last_clean":"2024-04-20T21:33:21.001834+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:33:21.001834+0000","last_undegraded":"2024-04-20T21:33:21.001834+0000","last_fullsized":"2024-04-20T21:33:21.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180412+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:33:21.180412+0000","last_peered":"2024-04-20T21:33:21.180412+0000","last_clean":"2024-04-20T21:33:21.180412+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:33:21.180412+0000","last_undegraded":"2024-04-20T21:33:21.180412+0000","last_fullsized":"2024-04-20T21:33:21.180412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210843+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:33:21.210843+0000","last_peered":"2024-04-20T21:33:21.210843+0000","last_clean":"2024-04-20T21:33:21.210843+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:33:21.210843+0000","last_undegraded":"2024-04-20T21:33:21.210843+0000","last_fullsized":"2024-04-20T21:33:21.210843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209230+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:33:21.209230+0000","last_peered":"2024-04-20T21:33:21.209230+0000","last_clean":"2024-04-20T21:33:21.209230+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:33:21.209230+0000","last_undegraded":"2024-04-20T21:33:21.209230+0000","last_fullsized":"2024-04-20T21:33:21.209230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208401+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:33:21.208401+0000","last_peered":"2024-04-20T21:33:21.208401+0000","last_clean":"2024-04-20T21:33:21.208401+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:33:21.208401+0000","last_undegraded":"2024-04-20T21:33:21.208401+0000","last_fullsized":"2024-04-20T21:33:21.208401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178827+0000","last_change":"2024-04-20T21:31:50.149839+0000","last_active":"2024-04-20T21:33:21.178827+0000","last_peered":"2024-04-20T21:33:21.178827+0000","last_clean":"2024-04-20T21:33:21.178827+0000","last_became_active":"2024-04-20T21:31:50.145443+0000","last_became_peered":"2024-04-20T21:31:50.145443+0000","last_unstale":"2024-04-20T21:33:21.178827+0000","last_undegraded":"2024-04-20T21:33:21.178827+0000","last_fullsized":"2024-04-20T21:33:21.178827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:56.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187531+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:33:21.187531+0000","last_peered":"2024-04-20T21:33:21.187531+0000","last_clean":"2024-04-20T21:33:21.187531+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:33:21.187531+0000","last_undegraded":"2024-04-20T21:33:21.187531+0000","last_fullsized":"2024-04-20T21:33:21.187531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.637271+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:33:21.637271+0000","last_peered":"2024-04-20T21:33:21.637271+0000","last_clean":"2024-04-20T21:33:21.637271+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:33:21.637271+0000","last_undegraded":"2024-04-20T21:33:21.637271+0000","last_fullsized":"2024-04-20T21:33:21.637271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208844+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:33:21.208844+0000","last_peered":"2024-04-20T21:33:21.208844+0000","last_clean":"2024-04-20T21:33:21.208844+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:33:21.208844+0000","last_undegraded":"2024-04-20T21:33:21.208844+0000","last_fullsized":"2024-04-20T21:33:21.208844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187478+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:33:21.187478+0000","last_peered":"2024-04-20T21:33:21.187478+0000","last_clean":"2024-04-20T21:33:21.187478+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:33:21.187478+0000","last_undegraded":"2024-04-20T21:33:21.187478+0000","last_fullsized":"2024-04-20T21:33:21.187478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209336+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:33:21.209336+0000","last_peered":"2024-04-20T21:33:21.209336+0000","last_clean":"2024-04-20T21:33:21.209336+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:33:21.209336+0000","last_undegraded":"2024-04-20T21:33:21.209336+0000","last_fullsized":"2024-04-20T21:33:21.209336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179012+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:33:21.179012+0000","last_peered":"2024-04-20T21:33:21.179012+0000","last_clean":"2024-04-20T21:33:21.179012+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:33:21.179012+0000","last_undegraded":"2024-04-20T21:33:21.179012+0000","last_fullsized":"2024-04-20T21:33:21.179012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187091+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:33:21.187091+0000","last_peered":"2024-04-20T21:33:21.187091+0000","last_clean":"2024-04-20T21:33:21.187091+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:33:21.187091+0000","last_undegraded":"2024-04-20T21:33:21.187091+0000","last_fullsized":"2024-04-20T21:33:21.187091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002448+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:33:21.002448+0000","last_peered":"2024-04-20T21:33:21.002448+0000","last_clean":"2024-04-20T21:33:21.002448+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:33:21.002448+0000","last_undegraded":"2024-04-20T21:33:21.002448+0000","last_fullsized":"2024-04-20T21:33:21.002448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.636310+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:33:21.636310+0000","last_peered":"2024-04-20T21:33:21.636310+0000","last_clean":"2024-04-20T21:33:21.636310+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:33:21.636310+0000","last_undegraded":"2024-04-20T21:33:21.636310+0000","last_fullsized":"2024-04-20T21:33:21.636310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208860+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:33:21.208860+0000","last_peered":"2024-04-20T21:33:21.208860+0000","last_clean":"2024-04-20T21:33:21.208860+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:33:21.208860+0000","last_undegraded":"2024-04-20T21:33:21.208860+0000","last_fullsized":"2024-04-20T21:33:21.208860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001730+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:33:21.001730+0000","last_peered":"2024-04-20T21:33:21.001730+0000","last_clean":"2024-04-20T21:33:21.001730+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:33:21.001730+0000","last_undegraded":"2024-04-20T21:33:21.001730+0000","last_fullsized":"2024-04-20T21:33:21.001730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210841+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:33:21.210841+0000","last_peered":"2024-04-20T21:33:21.210841+0000","last_clean":"2024-04-20T21:33:21.210841+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:33:21.210841+0000","last_undegraded":"2024-04-20T21:33:21.210841+0000","last_fullsized":"2024-04-20T21:33:21.210841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187622+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:33:21.187622+0000","last_peered":"2024-04-20T21:33:21.187622+0000","last_clean":"2024-04-20T21:33:21.187622+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:33:21.187622+0000","last_undegraded":"2024-04-20T21:33:21.187622+0000","last_fullsized":"2024-04-20T21:33:21.187622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208696+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:33:21.208696+0000","last_peered":"2024-04-20T21:33:21.208696+0000","last_clean":"2024-04-20T21:33:21.208696+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:33:21.208696+0000","last_undegraded":"2024-04-20T21:33:21.208696+0000","last_fullsized":"2024-04-20T21:33:21.208696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.644517+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:33:21.644517+0000","last_peered":"2024-04-20T21:33:21.644517+0000","last_clean":"2024-04-20T21:33:21.644517+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:33:21.644517+0000","last_undegraded":"2024-04-20T21:33:21.644517+0000","last_fullsized":"2024-04-20T21:33:21.644517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66528,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":300,"num_read_kb":215,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209092+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:33:21.209092+0000","last_peered":"2024-04-20T21:33:21.209092+0000","last_clean":"2024-04-20T21:33:21.209092+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:33:21.209092+0000","last_undegraded":"2024-04-20T21:33:21.209092+0000","last_fullsized":"2024-04-20T21:33:21.209092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003369+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:33:21.003369+0000","last_peered":"2024-04-20T21:33:21.003369+0000","last_clean":"2024-04-20T21:33:21.003369+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:33:21.003369+0000","last_undegraded":"2024-04-20T21:33:21.003369+0000","last_fullsized":"2024-04-20T21:33:21.003369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209293+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:33:21.209293+0000","last_peered":"2024-04-20T21:33:21.209293+0000","last_clean":"2024-04-20T21:33:21.209293+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:33:21.209293+0000","last_undegraded":"2024-04-20T21:33:21.209293+0000","last_fullsized":"2024-04-20T21:33:21.209293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187210+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:33:21.187210+0000","last_peered":"2024-04-20T21:33:21.187210+0000","last_clean":"2024-04-20T21:33:21.187210+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:33:21.187210+0000","last_undegraded":"2024-04-20T21:33:21.187210+0000","last_fullsized":"2024-04-20T21:33:21.187210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001392+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:33:21.001392+0000","last_peered":"2024-04-20T21:33:21.001392+0000","last_clean":"2024-04-20T21:33:21.001392+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:33:21.001392+0000","last_undegraded":"2024-04-20T21:33:21.001392+0000","last_fullsized":"2024-04-20T21:33:21.001392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.690778+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:33:21.690778+0000","last_peered":"2024-04-20T21:33:21.690778+0000","last_clean":"2024-04-20T21:33:21.690778+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:33:21.690778+0000","last_undegraded":"2024-04-20T21:33:21.690778+0000","last_fullsized":"2024-04-20T21:33:21.690778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000878+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:33:21.000878+0000","last_peered":"2024-04-20T21:33:21.000878+0000","last_clean":"2024-04-20T21:33:21.000878+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:33:21.000878+0000","last_undegraded":"2024-04-20T21:33:21.000878+0000","last_fullsized":"2024-04-20T21:33:21.000878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208879+0000","last_change":"2024-04-20T21:31:50.475713+0000","last_active":"2024-04-20T21:33:21.208879+0000","last_peered":"2024-04-20T21:33:21.208879+0000","last_clean":"2024-04-20T21:33:21.208879+0000","last_became_active":"2024-04-20T21:31:50.463790+0000","last_became_peered":"2024-04-20T21:31:50.463790+0000","last_unstale":"2024-04-20T21:33:21.208879+0000","last_undegraded":"2024-04-20T21:33:21.208879+0000","last_fullsized":"2024-04-20T21:33:21.208879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:30:43.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187021+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:33:21.187021+0000","last_peered":"2024-04-20T21:33:21.187021+0000","last_clean":"2024-04-20T21:33:21.187021+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:33:21.187021+0000","last_undegraded":"2024-04-20T21:33:21.187021+0000","last_fullsized":"2024-04-20T21:33:21.187021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209094+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:33:21.209094+0000","last_peered":"2024-04-20T21:33:21.209094+0000","last_clean":"2024-04-20T21:33:21.209094+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:33:21.209094+0000","last_undegraded":"2024-04-20T21:33:21.209094+0000","last_fullsized":"2024-04-20T21:33:21.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178972+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:33:21.178972+0000","last_peered":"2024-04-20T21:33:21.178972+0000","last_clean":"2024-04-20T21:33:21.178972+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:33:21.178972+0000","last_undegraded":"2024-04-20T21:33:21.178972+0000","last_fullsized":"2024-04-20T21:33:21.178972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001669+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:33:21.001669+0000","last_peered":"2024-04-20T21:33:21.001669+0000","last_clean":"2024-04-20T21:33:21.001669+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:33:21.001669+0000","last_undegraded":"2024-04-20T21:33:21.001669+0000","last_fullsized":"2024-04-20T21:33:21.001669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191442+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:33:21.191442+0000","last_peered":"2024-04-20T21:33:21.191442+0000","last_clean":"2024-04-20T21:33:21.191442+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:33:21.191442+0000","last_undegraded":"2024-04-20T21:33:21.191442+0000","last_fullsized":"2024-04-20T21:33:21.191442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001026+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:33:21.001026+0000","last_peered":"2024-04-20T21:33:21.001026+0000","last_clean":"2024-04-20T21:33:21.001026+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:33:21.001026+0000","last_undegraded":"2024-04-20T21:33:21.001026+0000","last_fullsized":"2024-04-20T21:33:21.001026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191331+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:33:21.191331+0000","last_peered":"2024-04-20T21:33:21.191331+0000","last_clean":"2024-04-20T21:33:21.191331+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:33:21.191331+0000","last_undegraded":"2024-04-20T21:33:21.191331+0000","last_fullsized":"2024-04-20T21:33:21.191331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187230+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:33:21.187230+0000","last_peered":"2024-04-20T21:33:21.187230+0000","last_clean":"2024-04-20T21:33:21.187230+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:33:21.187230+0000","last_undegraded":"2024-04-20T21:33:21.187230+0000","last_fullsized":"2024-04-20T21:33:21.187230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.396685+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:33:21.396685+0000","last_peered":"2024-04-20T21:33:21.396685+0000","last_clean":"2024-04-20T21:33:21.396685+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:33:21.396685+0000","last_undegraded":"2024-04-20T21:33:21.396685+0000","last_fullsized":"2024-04-20T21:33:21.396685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179616+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:33:21.179616+0000","last_peered":"2024-04-20T21:33:21.179616+0000","last_clean":"2024-04-20T21:33:21.179616+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:33:21.179616+0000","last_undegraded":"2024-04-20T21:33:21.179616+0000","last_fullsized":"2024-04-20T21:33:21.179616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290481+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:33:21.290481+0000","last_peered":"2024-04-20T21:33:21.290481+0000","last_clean":"2024-04-20T21:33:21.290481+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:33:21.290481+0000","last_undegraded":"2024-04-20T21:33:21.290481+0000","last_fullsized":"2024-04-20T21:33:21.290481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000909+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:33:21.000909+0000","last_peered":"2024-04-20T21:33:21.000909+0000","last_clean":"2024-04-20T21:33:21.000909+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:33:21.000909+0000","last_undegraded":"2024-04-20T21:33:21.000909+0000","last_fullsized":"2024-04-20T21:33:21.000909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209115+0000","last_change":"2024-04-20T21:31:50.152048+0000","last_active":"2024-04-20T21:33:21.209115+0000","last_peered":"2024-04-20T21:33:21.209115+0000","last_clean":"2024-04-20T21:33:21.209115+0000","last_became_active":"2024-04-20T21:31:50.151646+0000","last_became_peered":"2024-04-20T21:31:50.151646+0000","last_unstale":"2024-04-20T21:33:21.209115+0000","last_undegraded":"2024-04-20T21:33:21.209115+0000","last_fullsized":"2024-04-20T21:33:21.209115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:22.729679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001061+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:33:21.001061+0000","last_peered":"2024-04-20T21:33:21.001061+0000","last_clean":"2024-04-20T21:33:21.001061+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:33:21.001061+0000","last_undegraded":"2024-04-20T21:33:21.001061+0000","last_fullsized":"2024-04-20T21:33:21.001061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209167+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:33:21.209167+0000","last_peered":"2024-04-20T21:33:21.209167+0000","last_clean":"2024-04-20T21:33:21.209167+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:33:21.209167+0000","last_undegraded":"2024-04-20T21:33:21.209167+0000","last_fullsized":"2024-04-20T21:33:21.209167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209782+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:33:21.209782+0000","last_peered":"2024-04-20T21:33:21.209782+0000","last_clean":"2024-04-20T21:33:21.209782+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:33:21.209782+0000","last_undegraded":"2024-04-20T21:33:21.209782+0000","last_fullsized":"2024-04-20T21:33:21.209782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.345940+0000","last_change":"2024-04-20T21:31:50.154737+0000","last_active":"2024-04-20T21:33:21.345940+0000","last_peered":"2024-04-20T21:33:21.345940+0000","last_clean":"2024-04-20T21:33:21.345940+0000","last_became_active":"2024-04-20T21:31:50.145747+0000","last_became_peered":"2024-04-20T21:31:50.145747+0000","last_unstale":"2024-04-20T21:33:21.345940+0000","last_undegraded":"2024-04-20T21:33:21.345940+0000","last_fullsized":"2024-04-20T21:33:21.345940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:38.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290620+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:33:21.290620+0000","last_peered":"2024-04-20T21:33:21.290620+0000","last_clean":"2024-04-20T21:33:21.290620+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:33:21.290620+0000","last_undegraded":"2024-04-20T21:33:21.290620+0000","last_fullsized":"2024-04-20T21:33:21.290620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210644+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:33:21.210644+0000","last_peered":"2024-04-20T21:33:21.210644+0000","last_clean":"2024-04-20T21:33:21.210644+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:33:21.210644+0000","last_undegraded":"2024-04-20T21:33:21.210644+0000","last_fullsized":"2024-04-20T21:33:21.210644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208203+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:33:21.208203+0000","last_peered":"2024-04-20T21:33:21.208203+0000","last_clean":"2024-04-20T21:33:21.208203+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:33:21.208203+0000","last_undegraded":"2024-04-20T21:33:21.208203+0000","last_fullsized":"2024-04-20T21:33:21.208203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208737+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:33:21.208737+0000","last_peered":"2024-04-20T21:33:21.208737+0000","last_clean":"2024-04-20T21:33:21.208737+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:33:21.208737+0000","last_undegraded":"2024-04-20T21:33:21.208737+0000","last_fullsized":"2024-04-20T21:33:21.208737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179336+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:33:21.179336+0000","last_peered":"2024-04-20T21:33:21.179336+0000","last_clean":"2024-04-20T21:33:21.179336+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:33:21.179336+0000","last_undegraded":"2024-04-20T21:33:21.179336+0000","last_fullsized":"2024-04-20T21:33:21.179336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178794+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:33:21.178794+0000","last_peered":"2024-04-20T21:33:21.178794+0000","last_clean":"2024-04-20T21:33:21.178794+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:33:21.178794+0000","last_undegraded":"2024-04-20T21:33:21.178794+0000","last_fullsized":"2024-04-20T21:33:21.178794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003186+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:33:21.003186+0000","last_peered":"2024-04-20T21:33:21.003186+0000","last_clean":"2024-04-20T21:33:21.003186+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:33:21.003186+0000","last_undegraded":"2024-04-20T21:33:21.003186+0000","last_fullsized":"2024-04-20T21:33:21.003186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179630+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:33:21.179630+0000","last_peered":"2024-04-20T21:33:21.179630+0000","last_clean":"2024-04-20T21:33:21.179630+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:33:21.179630+0000","last_undegraded":"2024-04-20T21:33:21.179630+0000","last_fullsized":"2024-04-20T21:33:21.179630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208856+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:33:21.208856+0000","last_peered":"2024-04-20T21:33:21.208856+0000","last_clean":"2024-04-20T21:33:21.208856+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:33:21.208856+0000","last_undegraded":"2024-04-20T21:33:21.208856+0000","last_fullsized":"2024-04-20T21:33:21.208856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001256+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:33:21.001256+0000","last_peered":"2024-04-20T21:33:21.001256+0000","last_clean":"2024-04-20T21:33:21.001256+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:33:21.001256+0000","last_undegraded":"2024-04-20T21:33:21.001256+0000","last_fullsized":"2024-04-20T21:33:21.001256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003578+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:33:21.003578+0000","last_peered":"2024-04-20T21:33:21.003578+0000","last_clean":"2024-04-20T21:33:21.003578+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:33:21.003578+0000","last_undegraded":"2024-04-20T21:33:21.003578+0000","last_fullsized":"2024-04-20T21:33:21.003578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001746+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:33:21.001746+0000","last_peered":"2024-04-20T21:33:21.001746+0000","last_clean":"2024-04-20T21:33:21.001746+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:33:21.001746+0000","last_undegraded":"2024-04-20T21:33:21.001746+0000","last_fullsized":"2024-04-20T21:33:21.001746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.339251+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:33:21.339251+0000","last_peered":"2024-04-20T21:33:21.339251+0000","last_clean":"2024-04-20T21:33:21.339251+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:33:21.339251+0000","last_undegraded":"2024-04-20T21:33:21.339251+0000","last_fullsized":"2024-04-20T21:33:21.339251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001136+0000","last_change":"2024-04-20T21:31:50.155956+0000","last_active":"2024-04-20T21:33:21.001136+0000","last_peered":"2024-04-20T21:33:21.001136+0000","last_clean":"2024-04-20T21:33:21.001136+0000","last_became_active":"2024-04-20T21:31:50.152234+0000","last_became_peered":"2024-04-20T21:31:50.152234+0000","last_unstale":"2024-04-20T21:33:21.001136+0000","last_undegraded":"2024-04-20T21:33:21.001136+0000","last_fullsized":"2024-04-20T21:33:21.001136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:21.828525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209519+0000","last_change":"2024-04-20T21:31:50.158229+0000","last_active":"2024-04-20T21:33:21.209519+0000","last_peered":"2024-04-20T21:33:21.209519+0000","last_clean":"2024-04-20T21:33:21.209519+0000","last_became_active":"2024-04-20T21:31:50.154553+0000","last_became_peered":"2024-04-20T21:31:50.154553+0000","last_unstale":"2024-04-20T21:33:21.209519+0000","last_undegraded":"2024-04-20T21:33:21.209519+0000","last_fullsized":"2024-04-20T21:33:21.209519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:13:49.505978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001872+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:33:21.001872+0000","last_peered":"2024-04-20T21:33:21.001872+0000","last_clean":"2024-04-20T21:33:21.001872+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:33:21.001872+0000","last_undegraded":"2024-04-20T21:33:21.001872+0000","last_fullsized":"2024-04-20T21:33:21.001872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179016+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:33:21.179016+0000","last_peered":"2024-04-20T21:33:21.179016+0000","last_clean":"2024-04-20T21:33:21.179016+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:33:21.179016+0000","last_undegraded":"2024-04-20T21:33:21.179016+0000","last_fullsized":"2024-04-20T21:33:21.179016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003222+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:33:21.003222+0000","last_peered":"2024-04-20T21:33:21.003222+0000","last_clean":"2024-04-20T21:33:21.003222+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:33:21.003222+0000","last_undegraded":"2024-04-20T21:33:21.003222+0000","last_fullsized":"2024-04-20T21:33:21.003222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003065+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:33:21.003065+0000","last_peered":"2024-04-20T21:33:21.003065+0000","last_clean":"2024-04-20T21:33:21.003065+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:33:21.003065+0000","last_undegraded":"2024-04-20T21:33:21.003065+0000","last_fullsized":"2024-04-20T21:33:21.003065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210306+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:33:21.210306+0000","last_peered":"2024-04-20T21:33:21.210306+0000","last_clean":"2024-04-20T21:33:21.210306+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:33:21.210306+0000","last_undegraded":"2024-04-20T21:33:21.210306+0000","last_fullsized":"2024-04-20T21:33:21.210306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209836+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:33:21.209836+0000","last_peered":"2024-04-20T21:33:21.209836+0000","last_clean":"2024-04-20T21:33:21.209836+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:33:21.209836+0000","last_undegraded":"2024-04-20T21:33:21.209836+0000","last_fullsized":"2024-04-20T21:33:21.209836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179085+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:33:21.179085+0000","last_peered":"2024-04-20T21:33:21.179085+0000","last_clean":"2024-04-20T21:33:21.179085+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:33:21.179085+0000","last_undegraded":"2024-04-20T21:33:21.179085+0000","last_fullsized":"2024-04-20T21:33:21.179085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003456+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:33:21.003456+0000","last_peered":"2024-04-20T21:33:21.003456+0000","last_clean":"2024-04-20T21:33:21.003456+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:33:21.003456+0000","last_undegraded":"2024-04-20T21:33:21.003456+0000","last_fullsized":"2024-04-20T21:33:21.003456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180560+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:33:21.180560+0000","last_peered":"2024-04-20T21:33:21.180560+0000","last_clean":"2024-04-20T21:33:21.180560+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:33:21.180560+0000","last_undegraded":"2024-04-20T21:33:21.180560+0000","last_fullsized":"2024-04-20T21:33:21.180560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187015+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:33:21.187015+0000","last_peered":"2024-04-20T21:33:21.187015+0000","last_clean":"2024-04-20T21:33:21.187015+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:33:21.187015+0000","last_undegraded":"2024-04-20T21:33:21.187015+0000","last_fullsized":"2024-04-20T21:33:21.187015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179187+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:33:21.179187+0000","last_peered":"2024-04-20T21:33:21.179187+0000","last_clean":"2024-04-20T21:33:21.179187+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:33:21.179187+0000","last_undegraded":"2024-04-20T21:33:21.179187+0000","last_fullsized":"2024-04-20T21:33:21.179187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208484+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:33:21.208484+0000","last_peered":"2024-04-20T21:33:21.208484+0000","last_clean":"2024-04-20T21:33:21.208484+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:33:21.208484+0000","last_undegraded":"2024-04-20T21:33:21.208484+0000","last_fullsized":"2024-04-20T21:33:21.208484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209545+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:33:21.209545+0000","last_peered":"2024-04-20T21:33:21.209545+0000","last_clean":"2024-04-20T21:33:21.209545+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:33:21.209545+0000","last_undegraded":"2024-04-20T21:33:21.209545+0000","last_fullsized":"2024-04-20T21:33:21.209545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178975+0000","last_change":"2024-04-20T21:31:50.144697+0000","last_active":"2024-04-20T21:33:21.178975+0000","last_peered":"2024-04-20T21:33:21.178975+0000","last_clean":"2024-04-20T21:33:21.178975+0000","last_became_active":"2024-04-20T21:31:50.144294+0000","last_became_peered":"2024-04-20T21:31:50.144294+0000","last_unstale":"2024-04-20T21:33:21.178975+0000","last_undegraded":"2024-04-20T21:33:21.178975+0000","last_fullsized":"2024-04-20T21:33:21.178975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:35.702515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597501,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":993,"num_read_kb":607,"num_write":295,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1792503,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":459161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1142784,"data_stored":1258982,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":861628,"data_compressed":22071,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":468215,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1061128,"data_compressed":38743,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":467684,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":520011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":468955,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":530989,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":467878,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1192271,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134309,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201020,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:26.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 -- 172.21.15.7:0/1648534800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88102030 msgr2=0x7f7a88102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 --2- 172.21.15.7:0/1648534800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88102030 0x7f7a88102450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009900 tx=0x7f7a7802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 -- 172.21.15.7:0/1648534800 shutdown_connections 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 --2- 172.21.15.7:0/1648534800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88103220 0x7f7a881036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 --2- 172.21.15.7:0/1648534800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 0x7f7a8810a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 --2- 172.21.15.7:0/1648534800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88102030 0x7f7a88102450 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 -- 172.21.15.7:0/1648534800 >> 172.21.15.7:0/1648534800 conn(0x7f7a880fd580 msgr2=0x7f7a880ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 -- 172.21.15.7:0/1648534800 shutdown_connections 2024-04-20T21:33:26.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.978+0000 7f7a8da00640 1 -- 172.21.15.7:0/1648534800 wait complete. 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.979+0000 7f7a8da00640 1 Processor -- start 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 -- start start 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 0x7f7a8819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 0x7f7a881a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810dc50 con 0x7f7a88103220 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810dad0 con 0x7f7a88102030 2024-04-20T21:33:26.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a8da00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810ddd0 con 0x7f7a88103be0 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a86000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a86000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57446/0 (socket says 172.21.15.7:57446) 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a86000640 1 -- 172.21.15.7:0/2560904363 learned_addr learned my addr 172.21.15.7:0/2560904363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a86a00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 0x7f7a8819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.980+0000 7f7a87400640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 0x7f7a881a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 -- 172.21.15.7:0/2560904363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 msgr2=0x7f7a881a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 0x7f7a881a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 -- 172.21.15.7:0/2560904363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 msgr2=0x7f7a8819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 0x7f7a8819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 -- 172.21.15.7:0/2560904363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881a20a0 con 0x7f7a88103220 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86a00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 0x7f7a8819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a86000640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400e9f0 tx=0x7f7a7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7400cd80 con 0x7f7a88103220 2024-04-20T21:33:26.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a74010910 con 0x7f7a88103220 2024-04-20T21:33:26.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a881a2390 con 0x7f7a88103220 2024-04-20T21:33:26.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.981+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74018990 con 0x7f7a88103220 2024-04-20T21:33:26.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.982+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881a2930 con 0x7f7a88103220 2024-04-20T21:33:26.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.983+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a881a2b90 con 0x7f7a88103220 2024-04-20T21:33:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.986+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f7a74010ac0 con 0x7f7a88103220 2024-04-20T21:33:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.987+0000 7f7a7f400640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 0x7f7a6c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.987+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7a740d0d40 con 0x7f7a88103220 2024-04-20T21:33:26.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.988+0000 7f7a86a00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 0x7f7a6c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:26.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.988+0000 7f7a86a00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 0x7f7a6c0a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78005fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:26.992+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7a7409cb30 con 0x7f7a88103220 2024-04-20T21:33:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.200+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a880758b0 con 0x7f7a6c0a3ec0 2024-04-20T21:33:27.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.209+0000 7f7a7f400640 1 -- 172.21.15.7:0/2560904363 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257974 (secure 0 0 0) 0x7f7a880758b0 con 0x7f7a6c0a3ec0 2024-04-20T21:33:27.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 msgr2=0x7f7a6c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 0x7f7a6c0a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a78005fd0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 msgr2=0x7f7a8819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400e9f0 tx=0x7f7a7400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 shutdown_connections 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7a88103be0 0x7f7a881a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a88103220 0x7f7a8819a9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7a6c0a3ec0 0x7f7a6c0a6380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 --2- 172.21.15.7:0/2560904363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a88102030 0x7f7a8819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 >> 172.21.15.7:0/2560904363 conn(0x7f7a880fd580 msgr2=0x7f7a88074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.214+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 shutdown_connections 2024-04-20T21:33:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.215+0000 7f7a8da00640 1 -- 172.21.15.7:0/2560904363 wait complete. 2024-04-20T21:33:27.231 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-20T21:33:27.026702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188103,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1129,"num_read_kb":726,"num_write":478,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352708,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589116,"statfs":{"total":1151940427776,"available":1151579254784,"internally_reserved":0,"allocated":12128256,"data_stored":8216769,"data_compressed":165237,"data_compressed_allocated":212992,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.975837"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179119+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:33:21.179119+0000","last_peered":"2024-04-20T21:33:21.179119+0000","last_clean":"2024-04-20T21:33:21.179119+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:33:21.179119+0000","last_undegraded":"2024-04-20T21:33:21.179119+0000","last_fullsized":"2024-04-20T21:33:21.179119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210790+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:33:21.210790+0000","last_peered":"2024-04-20T21:33:21.210790+0000","last_clean":"2024-04-20T21:33:21.210790+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:33:21.210790+0000","last_undegraded":"2024-04-20T21:33:21.210790+0000","last_fullsized":"2024-04-20T21:33:21.210790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210576+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:33:21.210576+0000","last_peered":"2024-04-20T21:33:21.210576+0000","last_clean":"2024-04-20T21:33:21.210576+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:33:21.210576+0000","last_undegraded":"2024-04-20T21:33:21.210576+0000","last_fullsized":"2024-04-20T21:33:21.210576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290607+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:33:21.290607+0000","last_peered":"2024-04-20T21:33:21.290607+0000","last_clean":"2024-04-20T21:33:21.290607+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:33:21.290607+0000","last_undegraded":"2024-04-20T21:33:21.290607+0000","last_fullsized":"2024-04-20T21:33:21.290607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208826+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:33:21.208826+0000","last_peered":"2024-04-20T21:33:21.208826+0000","last_clean":"2024-04-20T21:33:21.208826+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:33:21.208826+0000","last_undegraded":"2024-04-20T21:33:21.208826+0000","last_fullsized":"2024-04-20T21:33:21.208826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000919+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:33:21.000919+0000","last_peered":"2024-04-20T21:33:21.000919+0000","last_clean":"2024-04-20T21:33:21.000919+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:33:21.000919+0000","last_undegraded":"2024-04-20T21:33:21.000919+0000","last_fullsized":"2024-04-20T21:33:21.000919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178892+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:33:21.178892+0000","last_peered":"2024-04-20T21:33:21.178892+0000","last_clean":"2024-04-20T21:33:21.178892+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:33:21.178892+0000","last_undegraded":"2024-04-20T21:33:21.178892+0000","last_fullsized":"2024-04-20T21:33:21.178892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208900+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:33:21.208900+0000","last_peered":"2024-04-20T21:33:21.208900+0000","last_clean":"2024-04-20T21:33:21.208900+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:33:21.208900+0000","last_undegraded":"2024-04-20T21:33:21.208900+0000","last_fullsized":"2024-04-20T21:33:21.208900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191648+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:33:21.191648+0000","last_peered":"2024-04-20T21:33:21.191648+0000","last_clean":"2024-04-20T21:33:21.191648+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:33:21.191648+0000","last_undegraded":"2024-04-20T21:33:21.191648+0000","last_fullsized":"2024-04-20T21:33:21.191648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001738+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:33:21.001738+0000","last_peered":"2024-04-20T21:33:21.001738+0000","last_clean":"2024-04-20T21:33:21.001738+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:33:21.001738+0000","last_undegraded":"2024-04-20T21:33:21.001738+0000","last_fullsized":"2024-04-20T21:33:21.001738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290577+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:33:21.290577+0000","last_peered":"2024-04-20T21:33:21.290577+0000","last_clean":"2024-04-20T21:33:21.290577+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:33:21.290577+0000","last_undegraded":"2024-04-20T21:33:21.290577+0000","last_fullsized":"2024-04-20T21:33:21.290577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209784+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:33:21.209784+0000","last_peered":"2024-04-20T21:33:21.209784+0000","last_clean":"2024-04-20T21:33:21.209784+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:33:21.209784+0000","last_undegraded":"2024-04-20T21:33:21.209784+0000","last_fullsized":"2024-04-20T21:33:21.209784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210061+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:33:21.210061+0000","last_peered":"2024-04-20T21:33:21.210061+0000","last_clean":"2024-04-20T21:33:21.210061+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:33:21.210061+0000","last_undegraded":"2024-04-20T21:33:21.210061+0000","last_fullsized":"2024-04-20T21:33:21.210061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210055+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:33:21.210055+0000","last_peered":"2024-04-20T21:33:21.210055+0000","last_clean":"2024-04-20T21:33:21.210055+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:33:21.210055+0000","last_undegraded":"2024-04-20T21:33:21.210055+0000","last_fullsized":"2024-04-20T21:33:21.210055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208278+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:33:21.208278+0000","last_peered":"2024-04-20T21:33:21.208278+0000","last_clean":"2024-04-20T21:33:21.208278+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:33:21.208278+0000","last_undegraded":"2024-04-20T21:33:21.208278+0000","last_fullsized":"2024-04-20T21:33:21.208278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179317+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:33:21.179317+0000","last_peered":"2024-04-20T21:33:21.179317+0000","last_clean":"2024-04-20T21:33:21.179317+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:33:21.179317+0000","last_undegraded":"2024-04-20T21:33:21.179317+0000","last_fullsized":"2024-04-20T21:33:21.179317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002145+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:33:21.002145+0000","last_peered":"2024-04-20T21:33:21.002145+0000","last_clean":"2024-04-20T21:33:21.002145+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:33:21.002145+0000","last_undegraded":"2024-04-20T21:33:21.002145+0000","last_fullsized":"2024-04-20T21:33:21.002145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003416+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:33:21.003416+0000","last_peered":"2024-04-20T21:33:21.003416+0000","last_clean":"2024-04-20T21:33:21.003416+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:33:21.003416+0000","last_undegraded":"2024-04-20T21:33:21.003416+0000","last_fullsized":"2024-04-20T21:33:21.003416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001834+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:33:21.001834+0000","last_peered":"2024-04-20T21:33:21.001834+0000","last_clean":"2024-04-20T21:33:21.001834+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:33:21.001834+0000","last_undegraded":"2024-04-20T21:33:21.001834+0000","last_fullsized":"2024-04-20T21:33:21.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180412+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:33:21.180412+0000","last_peered":"2024-04-20T21:33:21.180412+0000","last_clean":"2024-04-20T21:33:21.180412+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:33:21.180412+0000","last_undegraded":"2024-04-20T21:33:21.180412+0000","last_fullsized":"2024-04-20T21:33:21.180412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210843+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:33:21.210843+0000","last_peered":"2024-04-20T21:33:21.210843+0000","last_clean":"2024-04-20T21:33:21.210843+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:33:21.210843+0000","last_undegraded":"2024-04-20T21:33:21.210843+0000","last_fullsized":"2024-04-20T21:33:21.210843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209230+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:33:21.209230+0000","last_peered":"2024-04-20T21:33:21.209230+0000","last_clean":"2024-04-20T21:33:21.209230+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:33:21.209230+0000","last_undegraded":"2024-04-20T21:33:21.209230+0000","last_fullsized":"2024-04-20T21:33:21.209230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208401+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:33:21.208401+0000","last_peered":"2024-04-20T21:33:21.208401+0000","last_clean":"2024-04-20T21:33:21.208401+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:33:21.208401+0000","last_undegraded":"2024-04-20T21:33:21.208401+0000","last_fullsized":"2024-04-20T21:33:21.208401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178827+0000","last_change":"2024-04-20T21:31:50.149839+0000","last_active":"2024-04-20T21:33:21.178827+0000","last_peered":"2024-04-20T21:33:21.178827+0000","last_clean":"2024-04-20T21:33:21.178827+0000","last_became_active":"2024-04-20T21:31:50.145443+0000","last_became_peered":"2024-04-20T21:31:50.145443+0000","last_unstale":"2024-04-20T21:33:21.178827+0000","last_undegraded":"2024-04-20T21:33:21.178827+0000","last_fullsized":"2024-04-20T21:33:21.178827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:56.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187531+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:33:21.187531+0000","last_peered":"2024-04-20T21:33:21.187531+0000","last_clean":"2024-04-20T21:33:21.187531+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:33:21.187531+0000","last_undegraded":"2024-04-20T21:33:21.187531+0000","last_fullsized":"2024-04-20T21:33:21.187531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.637271+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:33:21.637271+0000","last_peered":"2024-04-20T21:33:21.637271+0000","last_clean":"2024-04-20T21:33:21.637271+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:33:21.637271+0000","last_undegraded":"2024-04-20T21:33:21.637271+0000","last_fullsized":"2024-04-20T21:33:21.637271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208844+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:33:21.208844+0000","last_peered":"2024-04-20T21:33:21.208844+0000","last_clean":"2024-04-20T21:33:21.208844+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:33:21.208844+0000","last_undegraded":"2024-04-20T21:33:21.208844+0000","last_fullsized":"2024-04-20T21:33:21.208844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187478+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:33:21.187478+0000","last_peered":"2024-04-20T21:33:21.187478+0000","last_clean":"2024-04-20T21:33:21.187478+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:33:21.187478+0000","last_undegraded":"2024-04-20T21:33:21.187478+0000","last_fullsized":"2024-04-20T21:33:21.187478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209336+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:33:21.209336+0000","last_peered":"2024-04-20T21:33:21.209336+0000","last_clean":"2024-04-20T21:33:21.209336+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:33:21.209336+0000","last_undegraded":"2024-04-20T21:33:21.209336+0000","last_fullsized":"2024-04-20T21:33:21.209336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179012+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:33:21.179012+0000","last_peered":"2024-04-20T21:33:21.179012+0000","last_clean":"2024-04-20T21:33:21.179012+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:33:21.179012+0000","last_undegraded":"2024-04-20T21:33:21.179012+0000","last_fullsized":"2024-04-20T21:33:21.179012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187091+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:33:21.187091+0000","last_peered":"2024-04-20T21:33:21.187091+0000","last_clean":"2024-04-20T21:33:21.187091+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:33:21.187091+0000","last_undegraded":"2024-04-20T21:33:21.187091+0000","last_fullsized":"2024-04-20T21:33:21.187091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002448+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:33:21.002448+0000","last_peered":"2024-04-20T21:33:21.002448+0000","last_clean":"2024-04-20T21:33:21.002448+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:33:21.002448+0000","last_undegraded":"2024-04-20T21:33:21.002448+0000","last_fullsized":"2024-04-20T21:33:21.002448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.636310+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:33:21.636310+0000","last_peered":"2024-04-20T21:33:21.636310+0000","last_clean":"2024-04-20T21:33:21.636310+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:33:21.636310+0000","last_undegraded":"2024-04-20T21:33:21.636310+0000","last_fullsized":"2024-04-20T21:33:21.636310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208860+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:33:21.208860+0000","last_peered":"2024-04-20T21:33:21.208860+0000","last_clean":"2024-04-20T21:33:21.208860+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:33:21.208860+0000","last_undegraded":"2024-04-20T21:33:21.208860+0000","last_fullsized":"2024-04-20T21:33:21.208860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001730+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:33:21.001730+0000","last_peered":"2024-04-20T21:33:21.001730+0000","last_clean":"2024-04-20T21:33:21.001730+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:33:21.001730+0000","last_undegraded":"2024-04-20T21:33:21.001730+0000","last_fullsized":"2024-04-20T21:33:21.001730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210841+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:33:21.210841+0000","last_peered":"2024-04-20T21:33:21.210841+0000","last_clean":"2024-04-20T21:33:21.210841+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:33:21.210841+0000","last_undegraded":"2024-04-20T21:33:21.210841+0000","last_fullsized":"2024-04-20T21:33:21.210841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187622+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:33:21.187622+0000","last_peered":"2024-04-20T21:33:21.187622+0000","last_clean":"2024-04-20T21:33:21.187622+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:33:21.187622+0000","last_undegraded":"2024-04-20T21:33:21.187622+0000","last_fullsized":"2024-04-20T21:33:21.187622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208696+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:33:21.208696+0000","last_peered":"2024-04-20T21:33:21.208696+0000","last_clean":"2024-04-20T21:33:21.208696+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:33:21.208696+0000","last_undegraded":"2024-04-20T21:33:21.208696+0000","last_fullsized":"2024-04-20T21:33:21.208696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.644517+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:33:21.644517+0000","last_peered":"2024-04-20T21:33:21.644517+0000","last_clean":"2024-04-20T21:33:21.644517+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:33:21.644517+0000","last_undegraded":"2024-04-20T21:33:21.644517+0000","last_fullsized":"2024-04-20T21:33:21.644517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66528,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":300,"num_read_kb":215,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209092+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:33:21.209092+0000","last_peered":"2024-04-20T21:33:21.209092+0000","last_clean":"2024-04-20T21:33:21.209092+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:33:21.209092+0000","last_undegraded":"2024-04-20T21:33:21.209092+0000","last_fullsized":"2024-04-20T21:33:21.209092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003369+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:33:21.003369+0000","last_peered":"2024-04-20T21:33:21.003369+0000","last_clean":"2024-04-20T21:33:21.003369+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:33:21.003369+0000","last_undegraded":"2024-04-20T21:33:21.003369+0000","last_fullsized":"2024-04-20T21:33:21.003369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209293+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:33:21.209293+0000","last_peered":"2024-04-20T21:33:21.209293+0000","last_clean":"2024-04-20T21:33:21.209293+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:33:21.209293+0000","last_undegraded":"2024-04-20T21:33:21.209293+0000","last_fullsized":"2024-04-20T21:33:21.209293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187210+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:33:21.187210+0000","last_peered":"2024-04-20T21:33:21.187210+0000","last_clean":"2024-04-20T21:33:21.187210+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:33:21.187210+0000","last_undegraded":"2024-04-20T21:33:21.187210+0000","last_fullsized":"2024-04-20T21:33:21.187210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001392+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:33:21.001392+0000","last_peered":"2024-04-20T21:33:21.001392+0000","last_clean":"2024-04-20T21:33:21.001392+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:33:21.001392+0000","last_undegraded":"2024-04-20T21:33:21.001392+0000","last_fullsized":"2024-04-20T21:33:21.001392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.690778+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:33:21.690778+0000","last_peered":"2024-04-20T21:33:21.690778+0000","last_clean":"2024-04-20T21:33:21.690778+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:33:21.690778+0000","last_undegraded":"2024-04-20T21:33:21.690778+0000","last_fullsized":"2024-04-20T21:33:21.690778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000878+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:33:21.000878+0000","last_peered":"2024-04-20T21:33:21.000878+0000","last_clean":"2024-04-20T21:33:21.000878+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:33:21.000878+0000","last_undegraded":"2024-04-20T21:33:21.000878+0000","last_fullsized":"2024-04-20T21:33:21.000878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208879+0000","last_change":"2024-04-20T21:31:50.475713+0000","last_active":"2024-04-20T21:33:21.208879+0000","last_peered":"2024-04-20T21:33:21.208879+0000","last_clean":"2024-04-20T21:33:21.208879+0000","last_became_active":"2024-04-20T21:31:50.463790+0000","last_became_peered":"2024-04-20T21:31:50.463790+0000","last_unstale":"2024-04-20T21:33:21.208879+0000","last_undegraded":"2024-04-20T21:33:21.208879+0000","last_fullsized":"2024-04-20T21:33:21.208879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:30:43.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187021+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:33:21.187021+0000","last_peered":"2024-04-20T21:33:21.187021+0000","last_clean":"2024-04-20T21:33:21.187021+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:33:21.187021+0000","last_undegraded":"2024-04-20T21:33:21.187021+0000","last_fullsized":"2024-04-20T21:33:21.187021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209094+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:33:21.209094+0000","last_peered":"2024-04-20T21:33:21.209094+0000","last_clean":"2024-04-20T21:33:21.209094+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:33:21.209094+0000","last_undegraded":"2024-04-20T21:33:21.209094+0000","last_fullsized":"2024-04-20T21:33:21.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178972+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:33:21.178972+0000","last_peered":"2024-04-20T21:33:21.178972+0000","last_clean":"2024-04-20T21:33:21.178972+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:33:21.178972+0000","last_undegraded":"2024-04-20T21:33:21.178972+0000","last_fullsized":"2024-04-20T21:33:21.178972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001669+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:33:21.001669+0000","last_peered":"2024-04-20T21:33:21.001669+0000","last_clean":"2024-04-20T21:33:21.001669+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:33:21.001669+0000","last_undegraded":"2024-04-20T21:33:21.001669+0000","last_fullsized":"2024-04-20T21:33:21.001669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191442+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:33:21.191442+0000","last_peered":"2024-04-20T21:33:21.191442+0000","last_clean":"2024-04-20T21:33:21.191442+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:33:21.191442+0000","last_undegraded":"2024-04-20T21:33:21.191442+0000","last_fullsized":"2024-04-20T21:33:21.191442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001026+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:33:21.001026+0000","last_peered":"2024-04-20T21:33:21.001026+0000","last_clean":"2024-04-20T21:33:21.001026+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:33:21.001026+0000","last_undegraded":"2024-04-20T21:33:21.001026+0000","last_fullsized":"2024-04-20T21:33:21.001026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191331+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:33:21.191331+0000","last_peered":"2024-04-20T21:33:21.191331+0000","last_clean":"2024-04-20T21:33:21.191331+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:33:21.191331+0000","last_undegraded":"2024-04-20T21:33:21.191331+0000","last_fullsized":"2024-04-20T21:33:21.191331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187230+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:33:21.187230+0000","last_peered":"2024-04-20T21:33:21.187230+0000","last_clean":"2024-04-20T21:33:21.187230+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:33:21.187230+0000","last_undegraded":"2024-04-20T21:33:21.187230+0000","last_fullsized":"2024-04-20T21:33:21.187230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.396685+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:33:21.396685+0000","last_peered":"2024-04-20T21:33:21.396685+0000","last_clean":"2024-04-20T21:33:21.396685+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:33:21.396685+0000","last_undegraded":"2024-04-20T21:33:21.396685+0000","last_fullsized":"2024-04-20T21:33:21.396685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179616+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:33:21.179616+0000","last_peered":"2024-04-20T21:33:21.179616+0000","last_clean":"2024-04-20T21:33:21.179616+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:33:21.179616+0000","last_undegraded":"2024-04-20T21:33:21.179616+0000","last_fullsized":"2024-04-20T21:33:21.179616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290481+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:33:21.290481+0000","last_peered":"2024-04-20T21:33:21.290481+0000","last_clean":"2024-04-20T21:33:21.290481+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:33:21.290481+0000","last_undegraded":"2024-04-20T21:33:21.290481+0000","last_fullsized":"2024-04-20T21:33:21.290481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000909+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:33:21.000909+0000","last_peered":"2024-04-20T21:33:21.000909+0000","last_clean":"2024-04-20T21:33:21.000909+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:33:21.000909+0000","last_undegraded":"2024-04-20T21:33:21.000909+0000","last_fullsized":"2024-04-20T21:33:21.000909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209115+0000","last_change":"2024-04-20T21:31:50.152048+0000","last_active":"2024-04-20T21:33:21.209115+0000","last_peered":"2024-04-20T21:33:21.209115+0000","last_clean":"2024-04-20T21:33:21.209115+0000","last_became_active":"2024-04-20T21:31:50.151646+0000","last_became_peered":"2024-04-20T21:31:50.151646+0000","last_unstale":"2024-04-20T21:33:21.209115+0000","last_undegraded":"2024-04-20T21:33:21.209115+0000","last_fullsized":"2024-04-20T21:33:21.209115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:22.729679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001061+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:33:21.001061+0000","last_peered":"2024-04-20T21:33:21.001061+0000","last_clean":"2024-04-20T21:33:21.001061+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:33:21.001061+0000","last_undegraded":"2024-04-20T21:33:21.001061+0000","last_fullsized":"2024-04-20T21:33:21.001061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209167+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:33:21.209167+0000","last_peered":"2024-04-20T21:33:21.209167+0000","last_clean":"2024-04-20T21:33:21.209167+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:33:21.209167+0000","last_undegraded":"2024-04-20T21:33:21.209167+0000","last_fullsized":"2024-04-20T21:33:21.209167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209782+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:33:21.209782+0000","last_peered":"2024-04-20T21:33:21.209782+0000","last_clean":"2024-04-20T21:33:21.209782+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:33:21.209782+0000","last_undegraded":"2024-04-20T21:33:21.209782+0000","last_fullsized":"2024-04-20T21:33:21.209782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.345940+0000","last_change":"2024-04-20T21:31:50.154737+0000","last_active":"2024-04-20T21:33:21.345940+0000","last_peered":"2024-04-20T21:33:21.345940+0000","last_clean":"2024-04-20T21:33:21.345940+0000","last_became_active":"2024-04-20T21:31:50.145747+0000","last_became_peered":"2024-04-20T21:31:50.145747+0000","last_unstale":"2024-04-20T21:33:21.345940+0000","last_undegraded":"2024-04-20T21:33:21.345940+0000","last_fullsized":"2024-04-20T21:33:21.345940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:38.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290620+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:33:21.290620+0000","last_peered":"2024-04-20T21:33:21.290620+0000","last_clean":"2024-04-20T21:33:21.290620+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:33:21.290620+0000","last_undegraded":"2024-04-20T21:33:21.290620+0000","last_fullsized":"2024-04-20T21:33:21.290620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210644+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:33:21.210644+0000","last_peered":"2024-04-20T21:33:21.210644+0000","last_clean":"2024-04-20T21:33:21.210644+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:33:21.210644+0000","last_undegraded":"2024-04-20T21:33:21.210644+0000","last_fullsized":"2024-04-20T21:33:21.210644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208203+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:33:21.208203+0000","last_peered":"2024-04-20T21:33:21.208203+0000","last_clean":"2024-04-20T21:33:21.208203+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:33:21.208203+0000","last_undegraded":"2024-04-20T21:33:21.208203+0000","last_fullsized":"2024-04-20T21:33:21.208203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208737+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:33:21.208737+0000","last_peered":"2024-04-20T21:33:21.208737+0000","last_clean":"2024-04-20T21:33:21.208737+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:33:21.208737+0000","last_undegraded":"2024-04-20T21:33:21.208737+0000","last_fullsized":"2024-04-20T21:33:21.208737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179336+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:33:21.179336+0000","last_peered":"2024-04-20T21:33:21.179336+0000","last_clean":"2024-04-20T21:33:21.179336+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:33:21.179336+0000","last_undegraded":"2024-04-20T21:33:21.179336+0000","last_fullsized":"2024-04-20T21:33:21.179336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178794+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:33:21.178794+0000","last_peered":"2024-04-20T21:33:21.178794+0000","last_clean":"2024-04-20T21:33:21.178794+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:33:21.178794+0000","last_undegraded":"2024-04-20T21:33:21.178794+0000","last_fullsized":"2024-04-20T21:33:21.178794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003186+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:33:21.003186+0000","last_peered":"2024-04-20T21:33:21.003186+0000","last_clean":"2024-04-20T21:33:21.003186+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:33:21.003186+0000","last_undegraded":"2024-04-20T21:33:21.003186+0000","last_fullsized":"2024-04-20T21:33:21.003186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179630+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:33:21.179630+0000","last_peered":"2024-04-20T21:33:21.179630+0000","last_clean":"2024-04-20T21:33:21.179630+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:33:21.179630+0000","last_undegraded":"2024-04-20T21:33:21.179630+0000","last_fullsized":"2024-04-20T21:33:21.179630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208856+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:33:21.208856+0000","last_peered":"2024-04-20T21:33:21.208856+0000","last_clean":"2024-04-20T21:33:21.208856+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:33:21.208856+0000","last_undegraded":"2024-04-20T21:33:21.208856+0000","last_fullsized":"2024-04-20T21:33:21.208856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001256+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:33:21.001256+0000","last_peered":"2024-04-20T21:33:21.001256+0000","last_clean":"2024-04-20T21:33:21.001256+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:33:21.001256+0000","last_undegraded":"2024-04-20T21:33:21.001256+0000","last_fullsized":"2024-04-20T21:33:21.001256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003578+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:33:21.003578+0000","last_peered":"2024-04-20T21:33:21.003578+0000","last_clean":"2024-04-20T21:33:21.003578+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:33:21.003578+0000","last_undegraded":"2024-04-20T21:33:21.003578+0000","last_fullsized":"2024-04-20T21:33:21.003578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001746+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:33:21.001746+0000","last_peered":"2024-04-20T21:33:21.001746+0000","last_clean":"2024-04-20T21:33:21.001746+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:33:21.001746+0000","last_undegraded":"2024-04-20T21:33:21.001746+0000","last_fullsized":"2024-04-20T21:33:21.001746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.339251+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:33:21.339251+0000","last_peered":"2024-04-20T21:33:21.339251+0000","last_clean":"2024-04-20T21:33:21.339251+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:33:21.339251+0000","last_undegraded":"2024-04-20T21:33:21.339251+0000","last_fullsized":"2024-04-20T21:33:21.339251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001136+0000","last_change":"2024-04-20T21:31:50.155956+0000","last_active":"2024-04-20T21:33:21.001136+0000","last_peered":"2024-04-20T21:33:21.001136+0000","last_clean":"2024-04-20T21:33:21.001136+0000","last_became_active":"2024-04-20T21:31:50.152234+0000","last_became_peered":"2024-04-20T21:31:50.152234+0000","last_unstale":"2024-04-20T21:33:21.001136+0000","last_undegraded":"2024-04-20T21:33:21.001136+0000","last_fullsized":"2024-04-20T21:33:21.001136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:21.828525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209519+0000","last_change":"2024-04-20T21:31:50.158229+0000","last_active":"2024-04-20T21:33:21.209519+0000","last_peered":"2024-04-20T21:33:21.209519+0000","last_clean":"2024-04-20T21:33:21.209519+0000","last_became_active":"2024-04-20T21:31:50.154553+0000","last_became_peered":"2024-04-20T21:31:50.154553+0000","last_unstale":"2024-04-20T21:33:21.209519+0000","last_undegraded":"2024-04-20T21:33:21.209519+0000","last_fullsized":"2024-04-20T21:33:21.209519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:13:49.505978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001872+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:33:21.001872+0000","last_peered":"2024-04-20T21:33:21.001872+0000","last_clean":"2024-04-20T21:33:21.001872+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:33:21.001872+0000","last_undegraded":"2024-04-20T21:33:21.001872+0000","last_fullsized":"2024-04-20T21:33:21.001872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179016+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:33:21.179016+0000","last_peered":"2024-04-20T21:33:21.179016+0000","last_clean":"2024-04-20T21:33:21.179016+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:33:21.179016+0000","last_undegraded":"2024-04-20T21:33:21.179016+0000","last_fullsized":"2024-04-20T21:33:21.179016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003222+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:33:21.003222+0000","last_peered":"2024-04-20T21:33:21.003222+0000","last_clean":"2024-04-20T21:33:21.003222+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:33:21.003222+0000","last_undegraded":"2024-04-20T21:33:21.003222+0000","last_fullsized":"2024-04-20T21:33:21.003222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003065+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:33:21.003065+0000","last_peered":"2024-04-20T21:33:21.003065+0000","last_clean":"2024-04-20T21:33:21.003065+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:33:21.003065+0000","last_undegraded":"2024-04-20T21:33:21.003065+0000","last_fullsized":"2024-04-20T21:33:21.003065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210306+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:33:21.210306+0000","last_peered":"2024-04-20T21:33:21.210306+0000","last_clean":"2024-04-20T21:33:21.210306+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:33:21.210306+0000","last_undegraded":"2024-04-20T21:33:21.210306+0000","last_fullsized":"2024-04-20T21:33:21.210306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209836+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:33:21.209836+0000","last_peered":"2024-04-20T21:33:21.209836+0000","last_clean":"2024-04-20T21:33:21.209836+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:33:21.209836+0000","last_undegraded":"2024-04-20T21:33:21.209836+0000","last_fullsized":"2024-04-20T21:33:21.209836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179085+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:33:21.179085+0000","last_peered":"2024-04-20T21:33:21.179085+0000","last_clean":"2024-04-20T21:33:21.179085+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:33:21.179085+0000","last_undegraded":"2024-04-20T21:33:21.179085+0000","last_fullsized":"2024-04-20T21:33:21.179085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003456+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:33:21.003456+0000","last_peered":"2024-04-20T21:33:21.003456+0000","last_clean":"2024-04-20T21:33:21.003456+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:33:21.003456+0000","last_undegraded":"2024-04-20T21:33:21.003456+0000","last_fullsized":"2024-04-20T21:33:21.003456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180560+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:33:21.180560+0000","last_peered":"2024-04-20T21:33:21.180560+0000","last_clean":"2024-04-20T21:33:21.180560+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:33:21.180560+0000","last_undegraded":"2024-04-20T21:33:21.180560+0000","last_fullsized":"2024-04-20T21:33:21.180560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187015+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:33:21.187015+0000","last_peered":"2024-04-20T21:33:21.187015+0000","last_clean":"2024-04-20T21:33:21.187015+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:33:21.187015+0000","last_undegraded":"2024-04-20T21:33:21.187015+0000","last_fullsized":"2024-04-20T21:33:21.187015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179187+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:33:21.179187+0000","last_peered":"2024-04-20T21:33:21.179187+0000","last_clean":"2024-04-20T21:33:21.179187+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:33:21.179187+0000","last_undegraded":"2024-04-20T21:33:21.179187+0000","last_fullsized":"2024-04-20T21:33:21.179187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208484+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:33:21.208484+0000","last_peered":"2024-04-20T21:33:21.208484+0000","last_clean":"2024-04-20T21:33:21.208484+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:33:21.208484+0000","last_undegraded":"2024-04-20T21:33:21.208484+0000","last_fullsized":"2024-04-20T21:33:21.208484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209545+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:33:21.209545+0000","last_peered":"2024-04-20T21:33:21.209545+0000","last_clean":"2024-04-20T21:33:21.209545+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:33:21.209545+0000","last_undegraded":"2024-04-20T21:33:21.209545+0000","last_fullsized":"2024-04-20T21:33:21.209545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178975+0000","last_change":"2024-04-20T21:31:50.144697+0000","last_active":"2024-04-20T21:33:21.178975+0000","last_peered":"2024-04-20T21:33:21.178975+0000","last_clean":"2024-04-20T21:33:21.178975+0000","last_became_active":"2024-04-20T21:31:50.144294+0000","last_became_peered":"2024-04-20T21:31:50.144294+0000","last_unstale":"2024-04-20T21:33:21.178975+0000","last_undegraded":"2024-04-20T21:33:21.178975+0000","last_fullsized":"2024-04-20T21:33:21.178975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:35.702515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597501,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":993,"num_read_kb":607,"num_write":295,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1792503,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":459161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1142784,"data_stored":1258982,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":861628,"data_compressed":22071,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":468215,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1061128,"data_compressed":38743,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":467684,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":520011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":468955,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":530989,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":467878,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1192271,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134309,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201020,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:27.235 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-20T21:33:27.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 -- 172.21.15.7:0/3053933346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 msgr2=0x7ff03c074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 --2- 172.21.15.7:0/3053933346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c074dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff028009900 tx=0x7ff02802bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 -- 172.21.15.7:0/3053933346 shutdown_connections 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 --2- 172.21.15.7:0/3053933346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c074dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 --2- 172.21.15.7:0/3053933346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 0x7ff03c10c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 --2- 172.21.15.7:0/3053933346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c074390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.467+0000 7ff040a00640 1 -- 172.21.15.7:0/3053933346 >> 172.21.15.7:0/3053933346 conn(0x7ff03c0fd590 msgr2=0x7ff03c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.468+0000 7ff040a00640 1 -- 172.21.15.7:0/3053933346 shutdown_connections 2024-04-20T21:33:27.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.468+0000 7ff040a00640 1 -- 172.21.15.7:0/3053933346 wait complete. 2024-04-20T21:33:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.469+0000 7ff040a00640 1 Processor -- start 2024-04-20T21:33:27.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.469+0000 7ff040a00640 1 -- start start 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 0x7ff03c1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c10fe90 con 0x7ff03c109eb0 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c10fd10 con 0x7ff03c074960 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff040a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c110010 con 0x7ff03c075f40 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff039e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff039e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57196/0 (socket says 172.21.15.7:57196) 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff039e00640 1 -- 172.21.15.7:0/4290272399 learned_addr learned my addr 172.21.15.7:0/4290272399 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff03a800640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 0x7ff03c1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.470+0000 7ff039400640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 -- 172.21.15.7:0/4290272399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 msgr2=0x7ff03c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 -- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 msgr2=0x7ff03c1a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 0x7ff03c1a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 -- 172.21.15.7:0/4290272399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c1a2060 con 0x7ff03c074960 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039400640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:27.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.471+0000 7ff039e00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e9f0 tx=0x7ff02c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.472+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c00cd80 con 0x7ff03c074960 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.472+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02c010910 con 0x7ff03c074960 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.472+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c018990 con 0x7ff03c074960 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.472+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c1a2350 con 0x7ff03c074960 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.472+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1a2820 con 0x7ff03c074960 2024-04-20T21:33:27.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.473+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c108c70 con 0x7ff03c074960 2024-04-20T21:33:27.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.475+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7ff02c010ac0 con 0x7ff03c074960 2024-04-20T21:33:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.477+0000 7ff032a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 0x7ff0200a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.477+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff02c0d0f70 con 0x7ff03c074960 2024-04-20T21:33:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.477+0000 7ff039400640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 0x7ff0200a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.478+0000 7ff039400640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 0x7ff0200a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff02802c2e0 tx=0x7ff02802c920 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.479+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff02c09ce20 con 0x7ff03c074960 2024-04-20T21:33:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.689+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff03c0623a0 con 0x7ff0200a3ec0 2024-04-20T21:33:27.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.702+0000 7ff032a00640 1 -- 172.21.15.7:0/4290272399 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257974 (secure 0 0 0) 0x7ff03c0623a0 con 0x7ff0200a3ec0 2024-04-20T21:33:27.704 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:27.705 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-20T21:33:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 msgr2=0x7ff0200a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 0x7ff0200a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff02802c2e0 tx=0x7ff02802c920 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 msgr2=0x7ff03c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00e9f0 tx=0x7ff02c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 shutdown_connections 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff03c109eb0 0x7ff03c1a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff03c075f40 0x7ff03c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff0200a3ec0 0x7ff0200a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 --2- 172.21.15.7:0/4290272399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff03c074960 0x7ff03c19a440 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 >> 172.21.15.7:0/4290272399 conn(0x7ff03c0fd590 msgr2=0x7ff03c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 shutdown_connections 2024-04-20T21:33:27.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.707+0000 7ff040a00640 1 -- 172.21.15.7:0/4290272399 wait complete. 2024-04-20T21:33:27.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-20T21:33:27.026702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188103,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1129,"num_read_kb":726,"num_write":478,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352708,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589116,"statfs":{"total":1151940427776,"available":1151579254784,"internally_reserved":0,"allocated":12128256,"data_stored":8216769,"data_compressed":165237,"data_compressed_allocated":212992,"data_compressed_original":2482176,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.975837"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179119+0000","last_change":"2024-04-20T21:30:52.943483+0000","last_active":"2024-04-20T21:33:21.179119+0000","last_peered":"2024-04-20T21:33:21.179119+0000","last_clean":"2024-04-20T21:33:21.179119+0000","last_became_active":"2024-04-20T21:30:52.943070+0000","last_became_peered":"2024-04-20T21:30:52.943070+0000","last_unstale":"2024-04-20T21:33:21.179119+0000","last_undegraded":"2024-04-20T21:33:21.179119+0000","last_fullsized":"2024-04-20T21:33:21.179119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:13.185561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210790+0000","last_change":"2024-04-20T21:30:52.934062+0000","last_active":"2024-04-20T21:33:21.210790+0000","last_peered":"2024-04-20T21:33:21.210790+0000","last_clean":"2024-04-20T21:33:21.210790+0000","last_became_active":"2024-04-20T21:30:52.933702+0000","last_became_peered":"2024-04-20T21:30:52.933702+0000","last_unstale":"2024-04-20T21:33:21.210790+0000","last_undegraded":"2024-04-20T21:33:21.210790+0000","last_fullsized":"2024-04-20T21:33:21.210790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:56:16.951360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210576+0000","last_change":"2024-04-20T21:30:52.933127+0000","last_active":"2024-04-20T21:33:21.210576+0000","last_peered":"2024-04-20T21:33:21.210576+0000","last_clean":"2024-04-20T21:33:21.210576+0000","last_became_active":"2024-04-20T21:30:52.931895+0000","last_became_peered":"2024-04-20T21:30:52.931895+0000","last_unstale":"2024-04-20T21:33:21.210576+0000","last_undegraded":"2024-04-20T21:33:21.210576+0000","last_fullsized":"2024-04-20T21:33:21.210576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:41:03.023171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290607+0000","last_change":"2024-04-20T21:30:52.931562+0000","last_active":"2024-04-20T21:33:21.290607+0000","last_peered":"2024-04-20T21:33:21.290607+0000","last_clean":"2024-04-20T21:33:21.290607+0000","last_became_active":"2024-04-20T21:30:52.926952+0000","last_became_peered":"2024-04-20T21:30:52.926952+0000","last_unstale":"2024-04-20T21:33:21.290607+0000","last_undegraded":"2024-04-20T21:33:21.290607+0000","last_fullsized":"2024-04-20T21:33:21.290607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.110994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208826+0000","last_change":"2024-04-20T21:30:52.936647+0000","last_active":"2024-04-20T21:33:21.208826+0000","last_peered":"2024-04-20T21:33:21.208826+0000","last_clean":"2024-04-20T21:33:21.208826+0000","last_became_active":"2024-04-20T21:30:52.936292+0000","last_became_peered":"2024-04-20T21:30:52.936292+0000","last_unstale":"2024-04-20T21:33:21.208826+0000","last_undegraded":"2024-04-20T21:33:21.208826+0000","last_fullsized":"2024-04-20T21:33:21.208826+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:18:10.105371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.930956+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.930558+0000","last_became_peered":"2024-04-20T21:30:52.930558+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:08:33.799769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000919+0000","last_change":"2024-04-20T21:30:52.929960+0000","last_active":"2024-04-20T21:33:21.000919+0000","last_peered":"2024-04-20T21:33:21.000919+0000","last_clean":"2024-04-20T21:33:21.000919+0000","last_became_active":"2024-04-20T21:30:52.929558+0000","last_became_peered":"2024-04-20T21:30:52.929558+0000","last_unstale":"2024-04-20T21:33:21.000919+0000","last_undegraded":"2024-04-20T21:33:21.000919+0000","last_fullsized":"2024-04-20T21:33:21.000919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:03:38.434868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178892+0000","last_change":"2024-04-20T21:30:52.942383+0000","last_active":"2024-04-20T21:33:21.178892+0000","last_peered":"2024-04-20T21:33:21.178892+0000","last_clean":"2024-04-20T21:33:21.178892+0000","last_became_active":"2024-04-20T21:30:52.941692+0000","last_became_peered":"2024-04-20T21:30:52.941692+0000","last_unstale":"2024-04-20T21:33:21.178892+0000","last_undegraded":"2024-04-20T21:33:21.178892+0000","last_fullsized":"2024-04-20T21:33:21.178892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:00:46.473084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208900+0000","last_change":"2024-04-20T21:30:52.925315+0000","last_active":"2024-04-20T21:33:21.208900+0000","last_peered":"2024-04-20T21:33:21.208900+0000","last_clean":"2024-04-20T21:33:21.208900+0000","last_became_active":"2024-04-20T21:30:52.924790+0000","last_became_peered":"2024-04-20T21:30:52.924790+0000","last_unstale":"2024-04-20T21:33:21.208900+0000","last_undegraded":"2024-04-20T21:33:21.208900+0000","last_fullsized":"2024-04-20T21:33:21.208900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:57.051959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191648+0000","last_change":"2024-04-20T21:30:52.932102+0000","last_active":"2024-04-20T21:33:21.191648+0000","last_peered":"2024-04-20T21:33:21.191648+0000","last_clean":"2024-04-20T21:33:21.191648+0000","last_became_active":"2024-04-20T21:30:52.931695+0000","last_became_peered":"2024-04-20T21:30:52.931695+0000","last_unstale":"2024-04-20T21:33:21.191648+0000","last_undegraded":"2024-04-20T21:33:21.191648+0000","last_fullsized":"2024-04-20T21:33:21.191648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:58.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001738+0000","last_change":"2024-04-20T21:30:52.921858+0000","last_active":"2024-04-20T21:33:21.001738+0000","last_peered":"2024-04-20T21:33:21.001738+0000","last_clean":"2024-04-20T21:33:21.001738+0000","last_became_active":"2024-04-20T21:30:52.921484+0000","last_became_peered":"2024-04-20T21:30:52.921484+0000","last_unstale":"2024-04-20T21:33:21.001738+0000","last_undegraded":"2024-04-20T21:33:21.001738+0000","last_fullsized":"2024-04-20T21:33:21.001738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:38.309584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290577+0000","last_change":"2024-04-20T21:30:52.921664+0000","last_active":"2024-04-20T21:33:21.290577+0000","last_peered":"2024-04-20T21:33:21.290577+0000","last_clean":"2024-04-20T21:33:21.290577+0000","last_became_active":"2024-04-20T21:30:52.920659+0000","last_became_peered":"2024-04-20T21:30:52.920659+0000","last_unstale":"2024-04-20T21:33:21.290577+0000","last_undegraded":"2024-04-20T21:33:21.290577+0000","last_fullsized":"2024-04-20T21:33:21.290577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:01:53.497809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209784+0000","last_change":"2024-04-20T21:30:52.931477+0000","last_active":"2024-04-20T21:33:21.209784+0000","last_peered":"2024-04-20T21:33:21.209784+0000","last_clean":"2024-04-20T21:33:21.209784+0000","last_became_active":"2024-04-20T21:30:52.931033+0000","last_became_peered":"2024-04-20T21:30:52.931033+0000","last_unstale":"2024-04-20T21:33:21.209784+0000","last_undegraded":"2024-04-20T21:33:21.209784+0000","last_fullsized":"2024-04-20T21:33:21.209784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:02:51.139314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210061+0000","last_change":"2024-04-20T21:30:52.939338+0000","last_active":"2024-04-20T21:33:21.210061+0000","last_peered":"2024-04-20T21:33:21.210061+0000","last_clean":"2024-04-20T21:33:21.210061+0000","last_became_active":"2024-04-20T21:30:52.935711+0000","last_became_peered":"2024-04-20T21:30:52.935711+0000","last_unstale":"2024-04-20T21:33:21.210061+0000","last_undegraded":"2024-04-20T21:33:21.210061+0000","last_fullsized":"2024-04-20T21:33:21.210061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:22.147407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210055+0000","last_change":"2024-04-20T21:30:52.944867+0000","last_active":"2024-04-20T21:33:21.210055+0000","last_peered":"2024-04-20T21:33:21.210055+0000","last_clean":"2024-04-20T21:33:21.210055+0000","last_became_active":"2024-04-20T21:30:52.937091+0000","last_became_peered":"2024-04-20T21:30:52.937091+0000","last_unstale":"2024-04-20T21:33:21.210055+0000","last_undegraded":"2024-04-20T21:33:21.210055+0000","last_fullsized":"2024-04-20T21:33:21.210055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.752970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208278+0000","last_change":"2024-04-20T21:30:52.933902+0000","last_active":"2024-04-20T21:33:21.208278+0000","last_peered":"2024-04-20T21:33:21.208278+0000","last_clean":"2024-04-20T21:33:21.208278+0000","last_became_active":"2024-04-20T21:30:52.933554+0000","last_became_peered":"2024-04-20T21:30:52.933554+0000","last_unstale":"2024-04-20T21:33:21.208278+0000","last_undegraded":"2024-04-20T21:33:21.208278+0000","last_fullsized":"2024-04-20T21:33:21.208278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:56.985551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179317+0000","last_change":"2024-04-20T21:30:52.943737+0000","last_active":"2024-04-20T21:33:21.179317+0000","last_peered":"2024-04-20T21:33:21.179317+0000","last_clean":"2024-04-20T21:33:21.179317+0000","last_became_active":"2024-04-20T21:30:52.943273+0000","last_became_peered":"2024-04-20T21:30:52.943273+0000","last_unstale":"2024-04-20T21:33:21.179317+0000","last_undegraded":"2024-04-20T21:33:21.179317+0000","last_fullsized":"2024-04-20T21:33:21.179317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:39:26.692320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002145+0000","last_change":"2024-04-20T21:30:52.929921+0000","last_active":"2024-04-20T21:33:21.002145+0000","last_peered":"2024-04-20T21:33:21.002145+0000","last_clean":"2024-04-20T21:33:21.002145+0000","last_became_active":"2024-04-20T21:30:52.924804+0000","last_became_peered":"2024-04-20T21:30:52.924804+0000","last_unstale":"2024-04-20T21:33:21.002145+0000","last_undegraded":"2024-04-20T21:33:21.002145+0000","last_fullsized":"2024-04-20T21:33:21.002145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:21:57.702979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003416+0000","last_change":"2024-04-20T21:30:52.945772+0000","last_active":"2024-04-20T21:33:21.003416+0000","last_peered":"2024-04-20T21:33:21.003416+0000","last_clean":"2024-04-20T21:33:21.003416+0000","last_became_active":"2024-04-20T21:30:52.945471+0000","last_became_peered":"2024-04-20T21:30:52.945471+0000","last_unstale":"2024-04-20T21:33:21.003416+0000","last_undegraded":"2024-04-20T21:33:21.003416+0000","last_fullsized":"2024-04-20T21:33:21.003416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:37.524139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001834+0000","last_change":"2024-04-20T21:30:52.379237+0000","last_active":"2024-04-20T21:33:21.001834+0000","last_peered":"2024-04-20T21:33:21.001834+0000","last_clean":"2024-04-20T21:33:21.001834+0000","last_became_active":"2024-04-20T21:30:52.378839+0000","last_became_peered":"2024-04-20T21:30:52.378839+0000","last_unstale":"2024-04-20T21:33:21.001834+0000","last_undegraded":"2024-04-20T21:33:21.001834+0000","last_fullsized":"2024-04-20T21:33:21.001834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:54:29.992430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180412+0000","last_change":"2024-04-20T21:30:52.946245+0000","last_active":"2024-04-20T21:33:21.180412+0000","last_peered":"2024-04-20T21:33:21.180412+0000","last_clean":"2024-04-20T21:33:21.180412+0000","last_became_active":"2024-04-20T21:30:52.945899+0000","last_became_peered":"2024-04-20T21:30:52.945899+0000","last_unstale":"2024-04-20T21:33:21.180412+0000","last_undegraded":"2024-04-20T21:33:21.180412+0000","last_fullsized":"2024-04-20T21:33:21.180412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:42:18.713237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.401603+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.401233+0000","last_became_peered":"2024-04-20T21:30:52.401233+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:23:48.835858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210843+0000","last_change":"2024-04-20T21:30:52.938839+0000","last_active":"2024-04-20T21:33:21.210843+0000","last_peered":"2024-04-20T21:33:21.210843+0000","last_clean":"2024-04-20T21:33:21.210843+0000","last_became_active":"2024-04-20T21:30:52.936188+0000","last_became_peered":"2024-04-20T21:30:52.936188+0000","last_unstale":"2024-04-20T21:33:21.210843+0000","last_undegraded":"2024-04-20T21:33:21.210843+0000","last_fullsized":"2024-04-20T21:33:21.210843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:36:51.410918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209230+0000","last_change":"2024-04-20T21:30:52.383505+0000","last_active":"2024-04-20T21:33:21.209230+0000","last_peered":"2024-04-20T21:33:21.209230+0000","last_clean":"2024-04-20T21:33:21.209230+0000","last_became_active":"2024-04-20T21:30:52.382865+0000","last_became_peered":"2024-04-20T21:30:52.382865+0000","last_unstale":"2024-04-20T21:33:21.209230+0000","last_undegraded":"2024-04-20T21:33:21.209230+0000","last_fullsized":"2024-04-20T21:33:21.209230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:03:04.115118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208401+0000","last_change":"2024-04-20T21:30:52.926525+0000","last_active":"2024-04-20T21:33:21.208401+0000","last_peered":"2024-04-20T21:33:21.208401+0000","last_clean":"2024-04-20T21:33:21.208401+0000","last_became_active":"2024-04-20T21:30:52.925985+0000","last_became_peered":"2024-04-20T21:30:52.925985+0000","last_unstale":"2024-04-20T21:33:21.208401+0000","last_undegraded":"2024-04-20T21:33:21.208401+0000","last_fullsized":"2024-04-20T21:33:21.208401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:49:39.662590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178827+0000","last_change":"2024-04-20T21:31:50.149839+0000","last_active":"2024-04-20T21:33:21.178827+0000","last_peered":"2024-04-20T21:33:21.178827+0000","last_clean":"2024-04-20T21:33:21.178827+0000","last_became_active":"2024-04-20T21:31:50.145443+0000","last_became_peered":"2024-04-20T21:31:50.145443+0000","last_unstale":"2024-04-20T21:33:21.178827+0000","last_undegraded":"2024-04-20T21:33:21.178827+0000","last_fullsized":"2024-04-20T21:33:21.178827+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:01:56.866470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187531+0000","last_change":"2024-04-20T21:30:52.928864+0000","last_active":"2024-04-20T21:33:21.187531+0000","last_peered":"2024-04-20T21:33:21.187531+0000","last_clean":"2024-04-20T21:33:21.187531+0000","last_became_active":"2024-04-20T21:30:52.928520+0000","last_became_peered":"2024-04-20T21:30:52.928520+0000","last_unstale":"2024-04-20T21:33:21.187531+0000","last_undegraded":"2024-04-20T21:33:21.187531+0000","last_fullsized":"2024-04-20T21:33:21.187531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:50.632635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.637271+0000","last_change":"2024-04-20T21:30:52.382706+0000","last_active":"2024-04-20T21:33:21.637271+0000","last_peered":"2024-04-20T21:33:21.637271+0000","last_clean":"2024-04-20T21:33:21.637271+0000","last_became_active":"2024-04-20T21:30:52.382366+0000","last_became_peered":"2024-04-20T21:30:52.382366+0000","last_unstale":"2024-04-20T21:33:21.637271+0000","last_undegraded":"2024-04-20T21:33:21.637271+0000","last_fullsized":"2024-04-20T21:33:21.637271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:48.661586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1855,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208844+0000","last_change":"2024-04-20T21:30:52.937568+0000","last_active":"2024-04-20T21:33:21.208844+0000","last_peered":"2024-04-20T21:33:21.208844+0000","last_clean":"2024-04-20T21:33:21.208844+0000","last_became_active":"2024-04-20T21:30:52.937153+0000","last_became_peered":"2024-04-20T21:30:52.937153+0000","last_unstale":"2024-04-20T21:33:21.208844+0000","last_undegraded":"2024-04-20T21:33:21.208844+0000","last_fullsized":"2024-04-20T21:33:21.208844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:27:37.464301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.190958+0000","last_change":"2024-04-20T21:30:52.386718+0000","last_active":"2024-04-20T21:33:21.190958+0000","last_peered":"2024-04-20T21:33:21.190958+0000","last_clean":"2024-04-20T21:33:21.190958+0000","last_became_active":"2024-04-20T21:30:52.386360+0000","last_became_peered":"2024-04-20T21:30:52.386360+0000","last_unstale":"2024-04-20T21:33:21.190958+0000","last_undegraded":"2024-04-20T21:33:21.190958+0000","last_fullsized":"2024-04-20T21:33:21.190958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:41.276476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187478+0000","last_change":"2024-04-20T21:30:52.946158+0000","last_active":"2024-04-20T21:33:21.187478+0000","last_peered":"2024-04-20T21:33:21.187478+0000","last_clean":"2024-04-20T21:33:21.187478+0000","last_became_active":"2024-04-20T21:30:52.945891+0000","last_became_peered":"2024-04-20T21:30:52.945891+0000","last_unstale":"2024-04-20T21:33:21.187478+0000","last_undegraded":"2024-04-20T21:33:21.187478+0000","last_fullsized":"2024-04-20T21:33:21.187478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.260548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209336+0000","last_change":"2024-04-20T21:30:52.391799+0000","last_active":"2024-04-20T21:33:21.209336+0000","last_peered":"2024-04-20T21:33:21.209336+0000","last_clean":"2024-04-20T21:33:21.209336+0000","last_became_active":"2024-04-20T21:30:52.391138+0000","last_became_peered":"2024-04-20T21:30:52.391138+0000","last_unstale":"2024-04-20T21:33:21.209336+0000","last_undegraded":"2024-04-20T21:33:21.209336+0000","last_fullsized":"2024-04-20T21:33:21.209336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:22.734507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179012+0000","last_change":"2024-04-20T21:30:52.934107+0000","last_active":"2024-04-20T21:33:21.179012+0000","last_peered":"2024-04-20T21:33:21.179012+0000","last_clean":"2024-04-20T21:33:21.179012+0000","last_became_active":"2024-04-20T21:30:52.933748+0000","last_became_peered":"2024-04-20T21:30:52.933748+0000","last_unstale":"2024-04-20T21:33:21.179012+0000","last_undegraded":"2024-04-20T21:33:21.179012+0000","last_fullsized":"2024-04-20T21:33:21.179012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:17.609179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187091+0000","last_change":"2024-04-20T21:30:52.378519+0000","last_active":"2024-04-20T21:33:21.187091+0000","last_peered":"2024-04-20T21:33:21.187091+0000","last_clean":"2024-04-20T21:33:21.187091+0000","last_became_active":"2024-04-20T21:30:52.378192+0000","last_became_peered":"2024-04-20T21:30:52.378192+0000","last_unstale":"2024-04-20T21:33:21.187091+0000","last_undegraded":"2024-04-20T21:33:21.187091+0000","last_fullsized":"2024-04-20T21:33:21.187091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:47:35.801360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.002448+0000","last_change":"2024-04-20T21:30:52.941852+0000","last_active":"2024-04-20T21:33:21.002448+0000","last_peered":"2024-04-20T21:33:21.002448+0000","last_clean":"2024-04-20T21:33:21.002448+0000","last_became_active":"2024-04-20T21:30:52.941412+0000","last_became_peered":"2024-04-20T21:30:52.941412+0000","last_unstale":"2024-04-20T21:33:21.002448+0000","last_undegraded":"2024-04-20T21:33:21.002448+0000","last_fullsized":"2024-04-20T21:33:21.002448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:38.488677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":171,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.636310+0000","last_change":"2024-04-20T21:30:52.382369+0000","last_active":"2024-04-20T21:33:21.636310+0000","last_peered":"2024-04-20T21:33:21.636310+0000","last_clean":"2024-04-20T21:33:21.636310+0000","last_became_active":"2024-04-20T21:30:52.381715+0000","last_became_peered":"2024-04-20T21:30:52.381715+0000","last_unstale":"2024-04-20T21:33:21.636310+0000","last_undegraded":"2024-04-20T21:33:21.636310+0000","last_fullsized":"2024-04-20T21:33:21.636310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:28.121654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208860+0000","last_change":"2024-04-20T21:30:52.919953+0000","last_active":"2024-04-20T21:33:21.208860+0000","last_peered":"2024-04-20T21:33:21.208860+0000","last_clean":"2024-04-20T21:33:21.208860+0000","last_became_active":"2024-04-20T21:30:52.919553+0000","last_became_peered":"2024-04-20T21:30:52.919553+0000","last_unstale":"2024-04-20T21:33:21.208860+0000","last_undegraded":"2024-04-20T21:33:21.208860+0000","last_fullsized":"2024-04-20T21:33:21.208860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:38:10.326478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001730+0000","last_change":"2024-04-20T21:30:52.382574+0000","last_active":"2024-04-20T21:33:21.001730+0000","last_peered":"2024-04-20T21:33:21.001730+0000","last_clean":"2024-04-20T21:33:21.001730+0000","last_became_active":"2024-04-20T21:30:52.380959+0000","last_became_peered":"2024-04-20T21:30:52.380959+0000","last_unstale":"2024-04-20T21:33:21.001730+0000","last_undegraded":"2024-04-20T21:33:21.001730+0000","last_fullsized":"2024-04-20T21:33:21.001730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:59.291402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210841+0000","last_change":"2024-04-20T21:30:52.944865+0000","last_active":"2024-04-20T21:33:21.210841+0000","last_peered":"2024-04-20T21:33:21.210841+0000","last_clean":"2024-04-20T21:33:21.210841+0000","last_became_active":"2024-04-20T21:30:52.944247+0000","last_became_peered":"2024-04-20T21:30:52.944247+0000","last_unstale":"2024-04-20T21:33:21.210841+0000","last_undegraded":"2024-04-20T21:33:21.210841+0000","last_fullsized":"2024-04-20T21:33:21.210841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:54.161838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187622+0000","last_change":"2024-04-20T21:30:52.381732+0000","last_active":"2024-04-20T21:33:21.187622+0000","last_peered":"2024-04-20T21:33:21.187622+0000","last_clean":"2024-04-20T21:33:21.187622+0000","last_became_active":"2024-04-20T21:30:52.381428+0000","last_became_peered":"2024-04-20T21:30:52.381428+0000","last_unstale":"2024-04-20T21:33:21.187622+0000","last_undegraded":"2024-04-20T21:33:21.187622+0000","last_fullsized":"2024-04-20T21:33:21.187622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:37.723096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208696+0000","last_change":"2024-04-20T21:30:52.924257+0000","last_active":"2024-04-20T21:33:21.208696+0000","last_peered":"2024-04-20T21:33:21.208696+0000","last_clean":"2024-04-20T21:33:21.208696+0000","last_became_active":"2024-04-20T21:30:52.923874+0000","last_became_peered":"2024-04-20T21:30:52.923874+0000","last_unstale":"2024-04-20T21:33:21.208696+0000","last_undegraded":"2024-04-20T21:33:21.208696+0000","last_fullsized":"2024-04-20T21:33:21.208696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:42.712957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'37","reported_seq":359,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.644517+0000","last_change":"2024-04-20T21:30:52.378694+0000","last_active":"2024-04-20T21:33:21.644517+0000","last_peered":"2024-04-20T21:33:21.644517+0000","last_clean":"2024-04-20T21:33:21.644517+0000","last_became_active":"2024-04-20T21:30:52.378326+0000","last_became_peered":"2024-04-20T21:30:52.378326+0000","last_unstale":"2024-04-20T21:33:21.644517+0000","last_undegraded":"2024-04-20T21:33:21.644517+0000","last_fullsized":"2024-04-20T21:33:21.644517+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:55.700608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66528,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":300,"num_read_kb":215,"num_write":38,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209092+0000","last_change":"2024-04-20T21:30:52.925703+0000","last_active":"2024-04-20T21:33:21.209092+0000","last_peered":"2024-04-20T21:33:21.209092+0000","last_clean":"2024-04-20T21:33:21.209092+0000","last_became_active":"2024-04-20T21:30:52.925202+0000","last_became_peered":"2024-04-20T21:30:52.925202+0000","last_unstale":"2024-04-20T21:33:21.209092+0000","last_undegraded":"2024-04-20T21:33:21.209092+0000","last_fullsized":"2024-04-20T21:33:21.209092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:20.189231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003369+0000","last_change":"2024-04-20T21:30:52.398203+0000","last_active":"2024-04-20T21:33:21.003369+0000","last_peered":"2024-04-20T21:33:21.003369+0000","last_clean":"2024-04-20T21:33:21.003369+0000","last_became_active":"2024-04-20T21:30:52.397811+0000","last_became_peered":"2024-04-20T21:30:52.397811+0000","last_unstale":"2024-04-20T21:33:21.003369+0000","last_undegraded":"2024-04-20T21:33:21.003369+0000","last_fullsized":"2024-04-20T21:33:21.003369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:28.682376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209293+0000","last_change":"2024-04-20T21:30:52.951623+0000","last_active":"2024-04-20T21:33:21.209293+0000","last_peered":"2024-04-20T21:33:21.209293+0000","last_clean":"2024-04-20T21:33:21.209293+0000","last_became_active":"2024-04-20T21:30:52.950303+0000","last_became_peered":"2024-04-20T21:30:52.950303+0000","last_unstale":"2024-04-20T21:33:21.209293+0000","last_undegraded":"2024-04-20T21:33:21.209293+0000","last_fullsized":"2024-04-20T21:33:21.209293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:31.401164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187210+0000","last_change":"2024-04-20T21:30:52.383509+0000","last_active":"2024-04-20T21:33:21.187210+0000","last_peered":"2024-04-20T21:33:21.187210+0000","last_clean":"2024-04-20T21:33:21.187210+0000","last_became_active":"2024-04-20T21:30:52.383179+0000","last_became_peered":"2024-04-20T21:30:52.383179+0000","last_unstale":"2024-04-20T21:33:21.187210+0000","last_undegraded":"2024-04-20T21:33:21.187210+0000","last_fullsized":"2024-04-20T21:33:21.187210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:02:20.181860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001392+0000","last_change":"2024-04-20T21:30:52.936640+0000","last_active":"2024-04-20T21:33:21.001392+0000","last_peered":"2024-04-20T21:33:21.001392+0000","last_clean":"2024-04-20T21:33:21.001392+0000","last_became_active":"2024-04-20T21:30:52.936239+0000","last_became_peered":"2024-04-20T21:30:52.936239+0000","last_unstale":"2024-04-20T21:33:21.001392+0000","last_undegraded":"2024-04-20T21:33:21.001392+0000","last_fullsized":"2024-04-20T21:33:21.001392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:59.300339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.690778+0000","last_change":"2024-04-20T21:30:52.380717+0000","last_active":"2024-04-20T21:33:21.690778+0000","last_peered":"2024-04-20T21:33:21.690778+0000","last_clean":"2024-04-20T21:33:21.690778+0000","last_became_active":"2024-04-20T21:30:52.379753+0000","last_became_peered":"2024-04-20T21:30:52.379753+0000","last_unstale":"2024-04-20T21:33:21.690778+0000","last_undegraded":"2024-04-20T21:33:21.690778+0000","last_fullsized":"2024-04-20T21:33:21.690778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:55:48.361275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000878+0000","last_change":"2024-04-20T21:30:52.928304+0000","last_active":"2024-04-20T21:33:21.000878+0000","last_peered":"2024-04-20T21:33:21.000878+0000","last_clean":"2024-04-20T21:33:21.000878+0000","last_became_active":"2024-04-20T21:30:52.927958+0000","last_became_peered":"2024-04-20T21:30:52.927958+0000","last_unstale":"2024-04-20T21:33:21.000878+0000","last_undegraded":"2024-04-20T21:33:21.000878+0000","last_fullsized":"2024-04-20T21:33:21.000878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:34:33.919630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208879+0000","last_change":"2024-04-20T21:31:50.475713+0000","last_active":"2024-04-20T21:33:21.208879+0000","last_peered":"2024-04-20T21:33:21.208879+0000","last_clean":"2024-04-20T21:33:21.208879+0000","last_became_active":"2024-04-20T21:31:50.463790+0000","last_became_peered":"2024-04-20T21:31:50.463790+0000","last_unstale":"2024-04-20T21:33:21.208879+0000","last_undegraded":"2024-04-20T21:33:21.208879+0000","last_fullsized":"2024-04-20T21:33:21.208879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:30:43.235787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187021+0000","last_change":"2024-04-20T21:30:52.936149+0000","last_active":"2024-04-20T21:33:21.187021+0000","last_peered":"2024-04-20T21:33:21.187021+0000","last_clean":"2024-04-20T21:33:21.187021+0000","last_became_active":"2024-04-20T21:30:52.935773+0000","last_became_peered":"2024-04-20T21:30:52.935773+0000","last_unstale":"2024-04-20T21:33:21.187021+0000","last_undegraded":"2024-04-20T21:33:21.187021+0000","last_fullsized":"2024-04-20T21:33:21.187021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:17.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209094+0000","last_change":"2024-04-20T21:30:52.377726+0000","last_active":"2024-04-20T21:33:21.209094+0000","last_peered":"2024-04-20T21:33:21.209094+0000","last_clean":"2024-04-20T21:33:21.209094+0000","last_became_active":"2024-04-20T21:30:52.377389+0000","last_became_peered":"2024-04-20T21:30:52.377389+0000","last_unstale":"2024-04-20T21:33:21.209094+0000","last_undegraded":"2024-04-20T21:33:21.209094+0000","last_fullsized":"2024-04-20T21:33:21.209094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:00:59.417204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178972+0000","last_change":"2024-04-20T21:30:52.927619+0000","last_active":"2024-04-20T21:33:21.178972+0000","last_peered":"2024-04-20T21:33:21.178972+0000","last_clean":"2024-04-20T21:33:21.178972+0000","last_became_active":"2024-04-20T21:30:52.927285+0000","last_became_peered":"2024-04-20T21:30:52.927285+0000","last_unstale":"2024-04-20T21:33:21.178972+0000","last_undegraded":"2024-04-20T21:33:21.178972+0000","last_fullsized":"2024-04-20T21:33:21.178972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:25:18.356059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001669+0000","last_change":"2024-04-20T21:30:52.388525+0000","last_active":"2024-04-20T21:33:21.001669+0000","last_peered":"2024-04-20T21:33:21.001669+0000","last_clean":"2024-04-20T21:33:21.001669+0000","last_became_active":"2024-04-20T21:30:52.388153+0000","last_became_peered":"2024-04-20T21:30:52.388153+0000","last_unstale":"2024-04-20T21:33:21.001669+0000","last_undegraded":"2024-04-20T21:33:21.001669+0000","last_fullsized":"2024-04-20T21:33:21.001669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:52:29.809869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191442+0000","last_change":"2024-04-20T21:30:52.944519+0000","last_active":"2024-04-20T21:33:21.191442+0000","last_peered":"2024-04-20T21:33:21.191442+0000","last_clean":"2024-04-20T21:33:21.191442+0000","last_became_active":"2024-04-20T21:30:52.944132+0000","last_became_peered":"2024-04-20T21:30:52.944132+0000","last_unstale":"2024-04-20T21:33:21.191442+0000","last_undegraded":"2024-04-20T21:33:21.191442+0000","last_fullsized":"2024-04-20T21:33:21.191442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:12:39.109875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001026+0000","last_change":"2024-04-20T21:30:52.387815+0000","last_active":"2024-04-20T21:33:21.001026+0000","last_peered":"2024-04-20T21:33:21.001026+0000","last_clean":"2024-04-20T21:33:21.001026+0000","last_became_active":"2024-04-20T21:30:52.387259+0000","last_became_peered":"2024-04-20T21:30:52.387259+0000","last_unstale":"2024-04-20T21:33:21.001026+0000","last_undegraded":"2024-04-20T21:33:21.001026+0000","last_fullsized":"2024-04-20T21:33:21.001026+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:43:30.473009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.191331+0000","last_change":"2024-04-20T21:30:52.936387+0000","last_active":"2024-04-20T21:33:21.191331+0000","last_peered":"2024-04-20T21:33:21.191331+0000","last_clean":"2024-04-20T21:33:21.191331+0000","last_became_active":"2024-04-20T21:30:52.936007+0000","last_became_peered":"2024-04-20T21:30:52.936007+0000","last_unstale":"2024-04-20T21:33:21.191331+0000","last_undegraded":"2024-04-20T21:33:21.191331+0000","last_fullsized":"2024-04-20T21:33:21.191331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:01:35.099505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187230+0000","last_change":"2024-04-20T21:30:52.929632+0000","last_active":"2024-04-20T21:33:21.187230+0000","last_peered":"2024-04-20T21:33:21.187230+0000","last_clean":"2024-04-20T21:33:21.187230+0000","last_became_active":"2024-04-20T21:30:52.928801+0000","last_became_peered":"2024-04-20T21:30:52.928801+0000","last_unstale":"2024-04-20T21:33:21.187230+0000","last_undegraded":"2024-04-20T21:33:21.187230+0000","last_fullsized":"2024-04-20T21:33:21.187230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:55:34.351120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.396685+0000","last_change":"2024-04-20T21:30:52.399753+0000","last_active":"2024-04-20T21:33:21.396685+0000","last_peered":"2024-04-20T21:33:21.396685+0000","last_clean":"2024-04-20T21:33:21.396685+0000","last_became_active":"2024-04-20T21:30:52.399283+0000","last_became_peered":"2024-04-20T21:30:52.399283+0000","last_unstale":"2024-04-20T21:33:21.396685+0000","last_undegraded":"2024-04-20T21:33:21.396685+0000","last_fullsized":"2024-04-20T21:33:21.396685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:42:50.438508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179616+0000","last_change":"2024-04-20T21:30:52.939268+0000","last_active":"2024-04-20T21:33:21.179616+0000","last_peered":"2024-04-20T21:33:21.179616+0000","last_clean":"2024-04-20T21:33:21.179616+0000","last_became_active":"2024-04-20T21:30:52.938777+0000","last_became_peered":"2024-04-20T21:30:52.938777+0000","last_unstale":"2024-04-20T21:33:21.179616+0000","last_undegraded":"2024-04-20T21:33:21.179616+0000","last_fullsized":"2024-04-20T21:33:21.179616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:25:46.716667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290481+0000","last_change":"2024-04-20T21:30:52.381333+0000","last_active":"2024-04-20T21:33:21.290481+0000","last_peered":"2024-04-20T21:33:21.290481+0000","last_clean":"2024-04-20T21:33:21.290481+0000","last_became_active":"2024-04-20T21:30:52.380992+0000","last_became_peered":"2024-04-20T21:30:52.380992+0000","last_unstale":"2024-04-20T21:33:21.290481+0000","last_undegraded":"2024-04-20T21:33:21.290481+0000","last_fullsized":"2024-04-20T21:33:21.290481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:39:56.006285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.000909+0000","last_change":"2024-04-20T21:29:31.376817+0000","last_active":"2024-04-20T21:33:21.000909+0000","last_peered":"2024-04-20T21:33:21.000909+0000","last_clean":"2024-04-20T21:33:21.000909+0000","last_became_active":"2024-04-20T21:29:31.376393+0000","last_became_peered":"2024-04-20T21:29:31.376393+0000","last_unstale":"2024-04-20T21:33:21.000909+0000","last_undegraded":"2024-04-20T21:33:21.000909+0000","last_fullsized":"2024-04-20T21:33:21.000909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:25:35.290797+0000","last_clean_scrub_stamp":"2024-04-20T21:25:35.290797+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:49.012248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209115+0000","last_change":"2024-04-20T21:31:50.152048+0000","last_active":"2024-04-20T21:33:21.209115+0000","last_peered":"2024-04-20T21:33:21.209115+0000","last_clean":"2024-04-20T21:33:21.209115+0000","last_became_active":"2024-04-20T21:31:50.151646+0000","last_became_peered":"2024-04-20T21:31:50.151646+0000","last_unstale":"2024-04-20T21:33:21.209115+0000","last_undegraded":"2024-04-20T21:33:21.209115+0000","last_fullsized":"2024-04-20T21:33:21.209115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:22.729679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001061+0000","last_change":"2024-04-20T21:30:52.383992+0000","last_active":"2024-04-20T21:33:21.001061+0000","last_peered":"2024-04-20T21:33:21.001061+0000","last_clean":"2024-04-20T21:33:21.001061+0000","last_became_active":"2024-04-20T21:30:52.383521+0000","last_became_peered":"2024-04-20T21:30:52.383521+0000","last_unstale":"2024-04-20T21:33:21.001061+0000","last_undegraded":"2024-04-20T21:33:21.001061+0000","last_fullsized":"2024-04-20T21:33:21.001061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:15:53.493139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209167+0000","last_change":"2024-04-20T21:30:52.389465+0000","last_active":"2024-04-20T21:33:21.209167+0000","last_peered":"2024-04-20T21:33:21.209167+0000","last_clean":"2024-04-20T21:33:21.209167+0000","last_became_active":"2024-04-20T21:30:52.389163+0000","last_became_peered":"2024-04-20T21:30:52.389163+0000","last_unstale":"2024-04-20T21:33:21.209167+0000","last_undegraded":"2024-04-20T21:33:21.209167+0000","last_fullsized":"2024-04-20T21:33:21.209167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:49.281566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209782+0000","last_change":"2024-04-20T21:30:52.935289+0000","last_active":"2024-04-20T21:33:21.209782+0000","last_peered":"2024-04-20T21:33:21.209782+0000","last_clean":"2024-04-20T21:33:21.209782+0000","last_became_active":"2024-04-20T21:30:52.934964+0000","last_became_peered":"2024-04-20T21:30:52.934964+0000","last_unstale":"2024-04-20T21:33:21.209782+0000","last_undegraded":"2024-04-20T21:33:21.209782+0000","last_fullsized":"2024-04-20T21:33:21.209782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:44:20.410173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'45","reported_seq":102,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.345940+0000","last_change":"2024-04-20T21:31:50.154737+0000","last_active":"2024-04-20T21:33:21.345940+0000","last_peered":"2024-04-20T21:33:21.345940+0000","last_clean":"2024-04-20T21:33:21.345940+0000","last_became_active":"2024-04-20T21:31:50.145747+0000","last_became_peered":"2024-04-20T21:31:50.145747+0000","last_unstale":"2024-04-20T21:33:21.345940+0000","last_undegraded":"2024-04-20T21:33:21.345940+0000","last_fullsized":"2024-04-20T21:33:21.345940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:38.751036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":81,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180472+0000","last_change":"2024-04-20T21:30:52.948832+0000","last_active":"2024-04-20T21:33:21.180472+0000","last_peered":"2024-04-20T21:33:21.180472+0000","last_clean":"2024-04-20T21:33:21.180472+0000","last_became_active":"2024-04-20T21:30:52.948491+0000","last_became_peered":"2024-04-20T21:30:52.948491+0000","last_unstale":"2024-04-20T21:33:21.180472+0000","last_undegraded":"2024-04-20T21:33:21.180472+0000","last_fullsized":"2024-04-20T21:33:21.180472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:34.612935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.290620+0000","last_change":"2024-04-20T21:30:52.387011+0000","last_active":"2024-04-20T21:33:21.290620+0000","last_peered":"2024-04-20T21:33:21.290620+0000","last_clean":"2024-04-20T21:33:21.290620+0000","last_became_active":"2024-04-20T21:30:52.386624+0000","last_became_peered":"2024-04-20T21:30:52.386624+0000","last_unstale":"2024-04-20T21:33:21.290620+0000","last_undegraded":"2024-04-20T21:33:21.290620+0000","last_fullsized":"2024-04-20T21:33:21.290620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:24:30.149486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210644+0000","last_change":"2024-04-20T21:30:52.950958+0000","last_active":"2024-04-20T21:33:21.210644+0000","last_peered":"2024-04-20T21:33:21.210644+0000","last_clean":"2024-04-20T21:33:21.210644+0000","last_became_active":"2024-04-20T21:30:52.950394+0000","last_became_peered":"2024-04-20T21:30:52.950394+0000","last_unstale":"2024-04-20T21:33:21.210644+0000","last_undegraded":"2024-04-20T21:33:21.210644+0000","last_fullsized":"2024-04-20T21:33:21.210644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:20:47.466225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208203+0000","last_change":"2024-04-20T21:30:52.398833+0000","last_active":"2024-04-20T21:33:21.208203+0000","last_peered":"2024-04-20T21:33:21.208203+0000","last_clean":"2024-04-20T21:33:21.208203+0000","last_became_active":"2024-04-20T21:30:52.398496+0000","last_became_peered":"2024-04-20T21:30:52.398496+0000","last_unstale":"2024-04-20T21:33:21.208203+0000","last_undegraded":"2024-04-20T21:33:21.208203+0000","last_fullsized":"2024-04-20T21:33:21.208203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:57.862927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208737+0000","last_change":"2024-04-20T21:30:52.925005+0000","last_active":"2024-04-20T21:33:21.208737+0000","last_peered":"2024-04-20T21:33:21.208737+0000","last_clean":"2024-04-20T21:33:21.208737+0000","last_became_active":"2024-04-20T21:30:52.924603+0000","last_became_peered":"2024-04-20T21:30:52.924603+0000","last_unstale":"2024-04-20T21:33:21.208737+0000","last_undegraded":"2024-04-20T21:33:21.208737+0000","last_fullsized":"2024-04-20T21:33:21.208737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:13.218400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179336+0000","last_change":"2024-04-20T21:30:52.404438+0000","last_active":"2024-04-20T21:33:21.179336+0000","last_peered":"2024-04-20T21:33:21.179336+0000","last_clean":"2024-04-20T21:33:21.179336+0000","last_became_active":"2024-04-20T21:30:52.404195+0000","last_became_peered":"2024-04-20T21:30:52.404195+0000","last_unstale":"2024-04-20T21:33:21.179336+0000","last_undegraded":"2024-04-20T21:33:21.179336+0000","last_fullsized":"2024-04-20T21:33:21.179336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:21:23.869768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178794+0000","last_change":"2024-04-20T21:30:52.935639+0000","last_active":"2024-04-20T21:33:21.178794+0000","last_peered":"2024-04-20T21:33:21.178794+0000","last_clean":"2024-04-20T21:33:21.178794+0000","last_became_active":"2024-04-20T21:30:52.935299+0000","last_became_peered":"2024-04-20T21:30:52.935299+0000","last_unstale":"2024-04-20T21:33:21.178794+0000","last_undegraded":"2024-04-20T21:33:21.178794+0000","last_fullsized":"2024-04-20T21:33:21.178794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:37.446127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003186+0000","last_change":"2024-04-20T21:30:52.398363+0000","last_active":"2024-04-20T21:33:21.003186+0000","last_peered":"2024-04-20T21:33:21.003186+0000","last_clean":"2024-04-20T21:33:21.003186+0000","last_became_active":"2024-04-20T21:30:52.397942+0000","last_became_peered":"2024-04-20T21:30:52.397942+0000","last_unstale":"2024-04-20T21:33:21.003186+0000","last_undegraded":"2024-04-20T21:33:21.003186+0000","last_fullsized":"2024-04-20T21:33:21.003186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:29.928712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179630+0000","last_change":"2024-04-20T21:30:52.942352+0000","last_active":"2024-04-20T21:33:21.179630+0000","last_peered":"2024-04-20T21:33:21.179630+0000","last_clean":"2024-04-20T21:33:21.179630+0000","last_became_active":"2024-04-20T21:30:52.941967+0000","last_became_peered":"2024-04-20T21:30:52.941967+0000","last_unstale":"2024-04-20T21:33:21.179630+0000","last_undegraded":"2024-04-20T21:33:21.179630+0000","last_fullsized":"2024-04-20T21:33:21.179630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:42.501558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208856+0000","last_change":"2024-04-20T21:30:52.383729+0000","last_active":"2024-04-20T21:33:21.208856+0000","last_peered":"2024-04-20T21:33:21.208856+0000","last_clean":"2024-04-20T21:33:21.208856+0000","last_became_active":"2024-04-20T21:30:52.383413+0000","last_became_peered":"2024-04-20T21:30:52.383413+0000","last_unstale":"2024-04-20T21:33:21.208856+0000","last_undegraded":"2024-04-20T21:33:21.208856+0000","last_fullsized":"2024-04-20T21:33:21.208856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:53:41.758190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001256+0000","last_change":"2024-04-20T21:30:52.930156+0000","last_active":"2024-04-20T21:33:21.001256+0000","last_peered":"2024-04-20T21:33:21.001256+0000","last_clean":"2024-04-20T21:33:21.001256+0000","last_became_active":"2024-04-20T21:30:52.929804+0000","last_became_peered":"2024-04-20T21:30:52.929804+0000","last_unstale":"2024-04-20T21:33:21.001256+0000","last_undegraded":"2024-04-20T21:33:21.001256+0000","last_fullsized":"2024-04-20T21:33:21.001256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:45:45.859436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003578+0000","last_change":"2024-04-20T21:30:52.397063+0000","last_active":"2024-04-20T21:33:21.003578+0000","last_peered":"2024-04-20T21:33:21.003578+0000","last_clean":"2024-04-20T21:33:21.003578+0000","last_became_active":"2024-04-20T21:30:52.396696+0000","last_became_peered":"2024-04-20T21:30:52.396696+0000","last_unstale":"2024-04-20T21:33:21.003578+0000","last_undegraded":"2024-04-20T21:33:21.003578+0000","last_fullsized":"2024-04-20T21:33:21.003578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:51.671903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001746+0000","last_change":"2024-04-20T21:30:52.924891+0000","last_active":"2024-04-20T21:33:21.001746+0000","last_peered":"2024-04-20T21:33:21.001746+0000","last_clean":"2024-04-20T21:33:21.001746+0000","last_became_active":"2024-04-20T21:30:52.924440+0000","last_became_peered":"2024-04-20T21:30:52.924440+0000","last_unstale":"2024-04-20T21:33:21.001746+0000","last_undegraded":"2024-04-20T21:33:21.001746+0000","last_fullsized":"2024-04-20T21:33:21.001746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:19.019121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.339251+0000","last_change":"2024-04-20T21:30:52.377384+0000","last_active":"2024-04-20T21:33:21.339251+0000","last_peered":"2024-04-20T21:33:21.339251+0000","last_clean":"2024-04-20T21:33:21.339251+0000","last_became_active":"2024-04-20T21:30:52.377098+0000","last_became_peered":"2024-04-20T21:30:52.377098+0000","last_unstale":"2024-04-20T21:33:21.339251+0000","last_undegraded":"2024-04-20T21:33:21.339251+0000","last_fullsized":"2024-04-20T21:33:21.339251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:25.318432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001136+0000","last_change":"2024-04-20T21:31:50.155956+0000","last_active":"2024-04-20T21:33:21.001136+0000","last_peered":"2024-04-20T21:33:21.001136+0000","last_clean":"2024-04-20T21:33:21.001136+0000","last_became_active":"2024-04-20T21:31:50.152234+0000","last_became_peered":"2024-04-20T21:31:50.152234+0000","last_unstale":"2024-04-20T21:33:21.001136+0000","last_undegraded":"2024-04-20T21:33:21.001136+0000","last_fullsized":"2024-04-20T21:33:21.001136+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:21.828525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209519+0000","last_change":"2024-04-20T21:31:50.158229+0000","last_active":"2024-04-20T21:33:21.209519+0000","last_peered":"2024-04-20T21:33:21.209519+0000","last_clean":"2024-04-20T21:33:21.209519+0000","last_became_active":"2024-04-20T21:31:50.154553+0000","last_became_peered":"2024-04-20T21:31:50.154553+0000","last_unstale":"2024-04-20T21:33:21.209519+0000","last_undegraded":"2024-04-20T21:33:21.209519+0000","last_fullsized":"2024-04-20T21:33:21.209519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:51.346384+0000","last_clean_scrub_stamp":"2024-04-20T21:30:51.346384+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:13:49.505978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.001872+0000","last_change":"2024-04-20T21:30:52.923351+0000","last_active":"2024-04-20T21:33:21.001872+0000","last_peered":"2024-04-20T21:33:21.001872+0000","last_clean":"2024-04-20T21:33:21.001872+0000","last_became_active":"2024-04-20T21:30:52.922965+0000","last_became_peered":"2024-04-20T21:30:52.922965+0000","last_unstale":"2024-04-20T21:33:21.001872+0000","last_undegraded":"2024-04-20T21:33:21.001872+0000","last_fullsized":"2024-04-20T21:33:21.001872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:03.167508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179016+0000","last_change":"2024-04-20T21:30:52.929157+0000","last_active":"2024-04-20T21:33:21.179016+0000","last_peered":"2024-04-20T21:33:21.179016+0000","last_clean":"2024-04-20T21:33:21.179016+0000","last_became_active":"2024-04-20T21:30:52.928802+0000","last_became_peered":"2024-04-20T21:30:52.928802+0000","last_unstale":"2024-04-20T21:33:21.179016+0000","last_undegraded":"2024-04-20T21:33:21.179016+0000","last_fullsized":"2024-04-20T21:33:21.179016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:24:58.192244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003222+0000","last_change":"2024-04-20T21:30:52.922144+0000","last_active":"2024-04-20T21:33:21.003222+0000","last_peered":"2024-04-20T21:33:21.003222+0000","last_clean":"2024-04-20T21:33:21.003222+0000","last_became_active":"2024-04-20T21:30:52.921773+0000","last_became_peered":"2024-04-20T21:30:52.921773+0000","last_unstale":"2024-04-20T21:33:21.003222+0000","last_undegraded":"2024-04-20T21:33:21.003222+0000","last_fullsized":"2024-04-20T21:33:21.003222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:10.931850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003065+0000","last_change":"2024-04-20T21:30:52.941414+0000","last_active":"2024-04-20T21:33:21.003065+0000","last_peered":"2024-04-20T21:33:21.003065+0000","last_clean":"2024-04-20T21:33:21.003065+0000","last_became_active":"2024-04-20T21:30:52.941131+0000","last_became_peered":"2024-04-20T21:30:52.941131+0000","last_unstale":"2024-04-20T21:33:21.003065+0000","last_undegraded":"2024-04-20T21:33:21.003065+0000","last_fullsized":"2024-04-20T21:33:21.003065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:40.026008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.210306+0000","last_change":"2024-04-20T21:30:52.945975+0000","last_active":"2024-04-20T21:33:21.210306+0000","last_peered":"2024-04-20T21:33:21.210306+0000","last_clean":"2024-04-20T21:33:21.210306+0000","last_became_active":"2024-04-20T21:30:52.940771+0000","last_became_peered":"2024-04-20T21:30:52.940771+0000","last_unstale":"2024-04-20T21:33:21.210306+0000","last_undegraded":"2024-04-20T21:33:21.210306+0000","last_fullsized":"2024-04-20T21:33:21.210306+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:17:32.021597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209836+0000","last_change":"2024-04-20T21:30:52.936740+0000","last_active":"2024-04-20T21:33:21.209836+0000","last_peered":"2024-04-20T21:33:21.209836+0000","last_clean":"2024-04-20T21:33:21.209836+0000","last_became_active":"2024-04-20T21:30:52.936364+0000","last_became_peered":"2024-04-20T21:30:52.936364+0000","last_unstale":"2024-04-20T21:33:21.209836+0000","last_undegraded":"2024-04-20T21:33:21.209836+0000","last_fullsized":"2024-04-20T21:33:21.209836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:33:00.613187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179085+0000","last_change":"2024-04-20T21:30:52.929283+0000","last_active":"2024-04-20T21:33:21.179085+0000","last_peered":"2024-04-20T21:33:21.179085+0000","last_clean":"2024-04-20T21:33:21.179085+0000","last_became_active":"2024-04-20T21:30:52.928938+0000","last_became_peered":"2024-04-20T21:30:52.928938+0000","last_unstale":"2024-04-20T21:33:21.179085+0000","last_undegraded":"2024-04-20T21:33:21.179085+0000","last_fullsized":"2024-04-20T21:33:21.179085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:52.662953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.003456+0000","last_change":"2024-04-20T21:30:52.921433+0000","last_active":"2024-04-20T21:33:21.003456+0000","last_peered":"2024-04-20T21:33:21.003456+0000","last_clean":"2024-04-20T21:33:21.003456+0000","last_became_active":"2024-04-20T21:30:52.920915+0000","last_became_peered":"2024-04-20T21:30:52.920915+0000","last_unstale":"2024-04-20T21:33:21.003456+0000","last_undegraded":"2024-04-20T21:33:21.003456+0000","last_fullsized":"2024-04-20T21:33:21.003456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:56.569834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.180560+0000","last_change":"2024-04-20T21:30:52.946674+0000","last_active":"2024-04-20T21:33:21.180560+0000","last_peered":"2024-04-20T21:33:21.180560+0000","last_clean":"2024-04-20T21:33:21.180560+0000","last_became_active":"2024-04-20T21:30:52.945850+0000","last_became_peered":"2024-04-20T21:30:52.945850+0000","last_unstale":"2024-04-20T21:33:21.180560+0000","last_undegraded":"2024-04-20T21:33:21.180560+0000","last_fullsized":"2024-04-20T21:33:21.180560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:21:54.526320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.187015+0000","last_change":"2024-04-20T21:30:52.946615+0000","last_active":"2024-04-20T21:33:21.187015+0000","last_peered":"2024-04-20T21:33:21.187015+0000","last_clean":"2024-04-20T21:33:21.187015+0000","last_became_active":"2024-04-20T21:30:52.946347+0000","last_became_peered":"2024-04-20T21:30:52.946347+0000","last_unstale":"2024-04-20T21:33:21.187015+0000","last_undegraded":"2024-04-20T21:33:21.187015+0000","last_fullsized":"2024-04-20T21:33:21.187015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:33:36.143341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.179187+0000","last_change":"2024-04-20T21:30:52.941644+0000","last_active":"2024-04-20T21:33:21.179187+0000","last_peered":"2024-04-20T21:33:21.179187+0000","last_clean":"2024-04-20T21:33:21.179187+0000","last_became_active":"2024-04-20T21:30:52.939649+0000","last_became_peered":"2024-04-20T21:30:52.939649+0000","last_unstale":"2024-04-20T21:33:21.179187+0000","last_undegraded":"2024-04-20T21:33:21.179187+0000","last_fullsized":"2024-04-20T21:33:21.179187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:09:22.629102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.208484+0000","last_change":"2024-04-20T21:30:52.942092+0000","last_active":"2024-04-20T21:33:21.208484+0000","last_peered":"2024-04-20T21:33:21.208484+0000","last_clean":"2024-04-20T21:33:21.208484+0000","last_became_active":"2024-04-20T21:30:52.941752+0000","last_became_peered":"2024-04-20T21:30:52.941752+0000","last_unstale":"2024-04-20T21:33:21.208484+0000","last_undegraded":"2024-04-20T21:33:21.208484+0000","last_fullsized":"2024-04-20T21:33:21.208484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:59.764599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.209545+0000","last_change":"2024-04-20T21:30:52.944069+0000","last_active":"2024-04-20T21:33:21.209545+0000","last_peered":"2024-04-20T21:33:21.209545+0000","last_clean":"2024-04-20T21:33:21.209545+0000","last_became_active":"2024-04-20T21:30:52.943577+0000","last_became_peered":"2024-04-20T21:30:52.943577+0000","last_unstale":"2024-04-20T21:33:21.209545+0000","last_undegraded":"2024-04-20T21:33:21.209545+0000","last_fullsized":"2024-04-20T21:33:21.209545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:52:54.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-20T21:33:21.178975+0000","last_change":"2024-04-20T21:31:50.144697+0000","last_active":"2024-04-20T21:33:21.178975+0000","last_peered":"2024-04-20T21:33:21.178975+0000","last_clean":"2024-04-20T21:33:21.178975+0000","last_became_active":"2024-04-20T21:31:50.144294+0000","last_became_peered":"2024-04-20T21:31:50.144294+0000","last_unstale":"2024-04-20T21:33:21.178975+0000","last_undegraded":"2024-04-20T21:33:21.178975+0000","last_fullsized":"2024-04-20T21:33:21.178975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:30:52.359930+0000","last_clean_scrub_stamp":"2024-04-20T21:30:52.359930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:38:35.702515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107454,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597501,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":993,"num_read_kb":607,"num_write":295,"num_write_kb":1442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":1792503,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":459161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1142784,"data_stored":1258982,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":861628,"data_compressed":22071,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":468215,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":1061128,"data_compressed":38743,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":467684,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":520011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":468955,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":530989,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149784,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":467878,"data_compressed":2925,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":459867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":1192271,"data_compressed":44899,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35818,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134309,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201020,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:33:27.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 -- 172.21.15.7:0/3242700767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105de0 msgr2=0x7f580010c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 --2- 172.21.15.7:0/3242700767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105de0 0x7f580010c670 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009ca0 tx=0x7f57f802c100 comp rx=0 tx=0).stop 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 -- 172.21.15.7:0/3242700767 shutdown_connections 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 --2- 172.21.15.7:0/3242700767 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105de0 0x7f580010c670 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 --2- 172.21.15.7:0/3242700767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105420 0x7f58001058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 --2- 172.21.15.7:0/3242700767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f5800104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 -- 172.21.15.7:0/3242700767 >> 172.21.15.7:0/3242700767 conn(0x7f58000ff7e0 msgr2=0x7f5800101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:27.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 -- 172.21.15.7:0/3242700767 shutdown_connections 2024-04-20T21:33:27.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.951+0000 7f5804a00640 1 -- 172.21.15.7:0/3242700767 wait complete. 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.952+0000 7f5804a00640 1 Processor -- start 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.952+0000 7f5804a00640 1 -- start start 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105420 0x7f580019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 0x7f58001a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f580010fe50 con 0x7f5800105de0 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f580010fcd0 con 0x7f5800104230 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f5804a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f580010ffd0 con 0x7f5800105420 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57218/0 (socket says 172.21.15.7:57218) 2024-04-20T21:33:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 -- 172.21.15.7:0/1736659472 learned_addr learned my addr 172.21.15.7:0/1736659472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fd400640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105420 0x7f580019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fe800640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 0x7f58001a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 -- 172.21.15.7:0/1736659472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105420 msgr2=0x7f580019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105420 0x7f580019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 -- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 msgr2=0x7f58001a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 0x7f58001a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fde00640 1 -- 172.21.15.7:0/1736659472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001a2040 con 0x7f5800104230 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.953+0000 7f57fe800640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 0x7f58001a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f57fde00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57f000e9c0 tx=0x7f57f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f000cd80 con 0x7f5800104230 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f00035f0 con 0x7f5800104230 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0010690 con 0x7f5800104230 2024-04-20T21:33:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001a2330 con 0x7f5800104230 2024-04-20T21:33:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.954+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a28d0 con 0x7f5800104230 2024-04-20T21:33:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.955+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580010b4f0 con 0x7f5800104230 2024-04-20T21:33:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.957+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f57f00037a0 con 0x7f5800104230 2024-04-20T21:33:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.960+0000 7f57f6a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 0x7f57e40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.960+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f57f00d0ee0 con 0x7f5800104230 2024-04-20T21:33:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.960+0000 7f57fd400640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 0x7f57e40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.961+0000 7f57fd400640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 0x7f57e40a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f580019b930 tx=0x7f57ec005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:27.963+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f57f009d320 con 0x7f5800104230 2024-04-20T21:33:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.179+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f580002d880 con 0x7f5800104230 2024-04-20T21:33:28.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.181+0000 7f57f6a00640 1 -- 172.21.15.7:0/1736659472 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22859 (secure 0 0 0) 0x7f57f009cc20 con 0x7f5800104230 2024-04-20T21:33:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:28.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:33:20.988355+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 msgr2=0x7f57e40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 0x7f57e40a6380 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f580019b930 tx=0x7f57ec005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 msgr2=0x7f580019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57f000e9c0 tx=0x7f57f000ee80 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 shutdown_connections 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5800105de0 0x7f58001a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5800105420 0x7f580019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57e40a3ec0 0x7f57e40a6380 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 --2- 172.21.15.7:0/1736659472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5800104230 0x7f580019a420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 >> 172.21.15.7:0/1736659472 conn(0x7f58000ff7e0 msgr2=0x7f58001086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 shutdown_connections 2024-04-20T21:33:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.184+0000 7f5804a00640 1 -- 172.21.15.7:0/1736659472 wait complete. 2024-04-20T21:33:28.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:33:28.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:33:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 -- 172.21.15.7:0/4143899994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4077b60 msgr2=0x7ffbe4077fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 --2- 172.21.15.7:0/4143899994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4077b60 0x7ffbe4077fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd002bc50 comp rx=0 tx=0).stop 2024-04-20T21:33:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 -- 172.21.15.7:0/4143899994 shutdown_connections 2024-04-20T21:33:28.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 --2- 172.21.15.7:0/4143899994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4077b60 0x7ffbe4077fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 --2- 172.21.15.7:0/4143899994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4078500 0x7ffbe4078da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 --2- 172.21.15.7:0/4143899994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4076920 0x7ffbe4076d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 -- 172.21.15.7:0/4143899994 >> 172.21.15.7:0/4143899994 conn(0x7ffbe40ff720 msgr2=0x7ffbe4101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 -- 172.21.15.7:0/4143899994 shutdown_connections 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.394+0000 7ffbea200640 1 -- 172.21.15.7:0/4143899994 wait complete. 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.396+0000 7ffbea200640 1 Processor -- start 2024-04-20T21:33:28.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.396+0000 7ffbea200640 1 -- start start 2024-04-20T21:33:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.396+0000 7ffbea200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4076920 0x7ffbe419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.396+0000 7ffbea200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4077b60 0x7ffbe419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.396+0000 7ffbea200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbea200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4114280 con 0x7ffbe4077b60 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbea200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4114100 con 0x7ffbe4076920 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbea200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe4114400 con 0x7ffbe4078500 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4077b60 0x7ffbe419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57048/0 (socket says 172.21.15.7:57048) 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 -- 172.21.15.7:0/3862695402 learned_addr learned my addr 172.21.15.7:0/3862695402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3400640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4076920 0x7ffbe419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 -- 172.21.15.7:0/3862695402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4076920 msgr2=0x7ffbe419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4076920 0x7ffbe419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 -- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4077b60 msgr2=0x7ffbe419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4077b60 0x7ffbe419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.397+0000 7ffbe3e00640 1 -- 172.21.15.7:0/3862695402 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe41a6410 con 0x7ffbe4078500 2024-04-20T21:33:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbe3e00640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009fd0 tx=0x7ffbdc00def0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc017070 con 0x7ffbe4078500 2024-04-20T21:33:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc003e50 con 0x7ffbe4078500 2024-04-20T21:33:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe41a6700 con 0x7ffbe4078500 2024-04-20T21:33:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc010460 con 0x7ffbe4078500 2024-04-20T21:33:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.398+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe41a6bd0 con 0x7ffbe4078500 2024-04-20T21:33:28.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.399+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe4110b20 con 0x7ffbe4078500 2024-04-20T21:33:28.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.401+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7ffbdc003970 con 0x7ffbe4078500 2024-04-20T21:33:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.403+0000 7ffbdbe00640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 0x7ffbc80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.403+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ffbdc0d0710 con 0x7ffbe4078500 2024-04-20T21:33:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.403+0000 7ffbe3400640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 0x7ffbc80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.404+0000 7ffbe3400640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 0x7ffbc80a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400abe0 tx=0x7ffbd400a2d0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.407+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffbdc09cb50 con 0x7ffbe4078500 2024-04-20T21:33:28.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:28.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:28.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:28 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1736659472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.619+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffbe4062b60 con 0x7ffbe4078500 2024-04-20T21:33:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.620+0000 7ffbdbe00640 1 -- 172.21.15.7:0/3862695402 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7ffbdc09c450 con 0x7ffbe4078500 2024-04-20T21:33:28.618 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 msgr2=0x7ffbc80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 0x7ffbc80a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400abe0 tx=0x7ffbd400a2d0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 msgr2=0x7ffbe41a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009fd0 tx=0x7ffbdc00def0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 shutdown_connections 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe4078500 0x7ffbe41a5da0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe4077b60 0x7ffbe419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffbc80a3e40 0x7ffbc80a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 --2- 172.21.15.7:0/3862695402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbe4076920 0x7ffbe419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 >> 172.21.15.7:0/3862695402 conn(0x7ffbe40ff720 msgr2=0x7ffbe4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.622+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 shutdown_connections 2024-04-20T21:33:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.623+0000 7ffbea200640 1 -- 172.21.15.7:0/3862695402 wait complete. 2024-04-20T21:33:28.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:28.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:28 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1736659472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='client.25343 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:33:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:28.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:28 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1736659472' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:28.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 -- 172.21.15.7:0/2882412924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 msgr2=0x7f75ec10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 --2- 172.21.15.7:0/2882412924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec10a3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009900 tx=0x7f75e402bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:28.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 -- 172.21.15.7:0/2882412924 shutdown_connections 2024-04-20T21:33:28.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 --2- 172.21.15.7:0/2882412924 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec10a3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 --2- 172.21.15.7:0/2882412924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec1031a0 0x7f75ec103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 --2- 172.21.15.7:0/2882412924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec101fb0 0x7f75ec1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 -- 172.21.15.7:0/2882412924 >> 172.21.15.7:0/2882412924 conn(0x7f75ec0fd520 msgr2=0x7f75ec0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:28.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 -- 172.21.15.7:0/2882412924 shutdown_connections 2024-04-20T21:33:28.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.827+0000 7f75f4200640 1 -- 172.21.15.7:0/2882412924 wait complete. 2024-04-20T21:33:28.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.828+0000 7f75f4200640 1 Processor -- start 2024-04-20T21:33:28.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 -- start start 2024-04-20T21:33:28.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec101fb0 0x7f75ec19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec1031a0 0x7f75ec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec10dc50 con 0x7f75ec101fb0 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec10dad0 con 0x7f75ec1031a0 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f4200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec10ddd0 con 0x7f75ec103b60 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f2200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f2200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57070/0 (socket says 172.21.15.7:57070) 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f2200640 1 -- 172.21.15.7:0/3349372939 learned_addr learned my addr 172.21.15.7:0/3349372939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f1800640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec101fb0 0x7f75ec19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.829+0000 7f75f0e00640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec1031a0 0x7f75ec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 -- 172.21.15.7:0/3349372939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec1031a0 msgr2=0x7f75ec19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec1031a0 0x7f75ec19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 -- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec101fb0 msgr2=0x7f75ec19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:28.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec101fb0 0x7f75ec19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 -- 172.21.15.7:0/3349372939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec1a2040 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75f2200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c180 tx=0x7f75e402c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4041070 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.830+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e40054e0 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.831+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec1a22d0 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.831+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4034440 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.831+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec1a27c0 con 0x7f75ec103b60 2024-04-20T21:33:28.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.832+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec1a2a20 con 0x7f75ec103b60 2024-04-20T21:33:28.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.834+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f75e4044070 con 0x7f75ec103b60 2024-04-20T21:33:28.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.835+0000 7f75ea000640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 0x7f75d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:28.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.835+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f75e40f2520 con 0x7f75ec103b60 2024-04-20T21:33:28.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.836+0000 7f75f1800640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 0x7f75d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:28.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.836+0000 7f75f1800640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 0x7f75d00a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75ec103000 tx=0x7f75dc005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:28.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:28.839+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f75e40be420 con 0x7f75ec103b60 2024-04-20T21:33:29.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.049+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75ec1a2db0 con 0x7f75ec103b60 2024-04-20T21:33:29.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.049+0000 7f75ea000640 1 -- 172.21.15.7:0/3349372939 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f75e40c2720 con 0x7f75ec103b60 2024-04-20T21:33:29.048 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 msgr2=0x7f75d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 0x7f75d00a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75ec103000 tx=0x7f75dc005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 msgr2=0x7f75ec1a19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75e402c180 tx=0x7f75e402c660 comp rx=0 tx=0).stop 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 shutdown_connections 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f75d00a3e40 0x7f75d00a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75ec1031a0 0x7f75ec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ec103b60 0x7f75ec1a19d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 --2- 172.21.15.7:0/3349372939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec101fb0 0x7f75ec19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 >> 172.21.15.7:0/3349372939 conn(0x7f75ec0fd520 msgr2=0x7f75ec074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 shutdown_connections 2024-04-20T21:33:29.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.052+0000 7f75f4200640 1 -- 172.21.15.7:0/3349372939 wait complete. 2024-04-20T21:33:29.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:33:29.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 -- 172.21.15.7:0/2908721699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27781041b0 msgr2=0x7f27781045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 --2- 172.21.15.7:0/2908721699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27781041b0 0x7f27781045d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2768009900 tx=0x7f276802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 -- 172.21.15.7:0/2908721699 shutdown_connections 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 --2- 172.21.15.7:0/2908721699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781053a0 0x7f2778105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 --2- 172.21.15.7:0/2908721699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2778105d60 0x7f277810c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 --2- 172.21.15.7:0/2908721699 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27781041b0 0x7f27781045d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.255+0000 7f277e000640 1 -- 172.21.15.7:0/2908721699 >> 172.21.15.7:0/2908721699 conn(0x7f27780ff720 msgr2=0x7f2778101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.256+0000 7f277e000640 1 -- 172.21.15.7:0/2908721699 shutdown_connections 2024-04-20T21:33:29.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.256+0000 7f277e000640 1 -- 172.21.15.7:0/2908721699 wait complete. 2024-04-20T21:33:29.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 Processor -- start 2024-04-20T21:33:29.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 -- start start 2024-04-20T21:33:29.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 0x7f277819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 0x7f277819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2778105d60 0x7f27781a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f277810fe60 con 0x7f27781053a0 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f277810fce0 con 0x7f27781041b0 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f277e000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f277810ffe0 con 0x7f2778105d60 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f276fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 0x7f277819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f2777400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 0x7f277819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f276fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 0x7f277819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57530/0 (socket says 172.21.15.7:57530) 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.257+0000 7f276fe00640 1 -- 172.21.15.7:0/556933149 learned_addr learned my addr 172.21.15.7:0/556933149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777e00640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2778105d60 0x7f27781a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 -- 172.21.15.7:0/556933149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2778105d60 msgr2=0x7f27781a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2778105d60 0x7f27781a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 -- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 msgr2=0x7f277819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 0x7f277819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 -- 172.21.15.7:0/556933149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781a20d0 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.258+0000 7f2777400640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 0x7f277819a4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2768009230 tx=0x7f276802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.259+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768043020 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.259+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27680054e0 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.259+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27781a2360 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.259+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768003b60 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.259+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781a28d0 con 0x7f27781041b0 2024-04-20T21:33:29.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.260+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277810b550 con 0x7f27781041b0 2024-04-20T21:33:29.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.263+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f2768046020 con 0x7f27781041b0 2024-04-20T21:33:29.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.264+0000 7f2774c00640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 0x7f275c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.264+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f27680f29a0 con 0x7f27781041b0 2024-04-20T21:33:29.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.264+0000 7f276fe00640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 0x7f275c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.265+0000 7f276fe00640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 0x7f275c0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f277819b9c0 tx=0x7f2760005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:29.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.268+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f276804d020 con 0x7f27781041b0 2024-04-20T21:33:29.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.479+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2778062b60 con 0x7f27781041b0 2024-04-20T21:33:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.479+0000 7f2774c00640 1 -- 172.21.15.7:0/556933149 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f27680beef0 con 0x7f27781041b0 2024-04-20T21:33:29.478 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:33:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 msgr2=0x7f275c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 0x7f275c0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f277819b9c0 tx=0x7f2760005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 msgr2=0x7f277819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 0x7f277819a4b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2768009230 tx=0x7f276802c5d0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 shutdown_connections 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2778105d60 0x7f27781a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27781053a0 0x7f277819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f275c0a3ec0 0x7f275c0a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 --2- 172.21.15.7:0/556933149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27781041b0 0x7f277819a4b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 >> 172.21.15.7:0/556933149 conn(0x7f27780ff720 msgr2=0x7f2778108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 shutdown_connections 2024-04-20T21:33:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.482+0000 7f277e000640 1 -- 172.21.15.7:0/556933149 wait complete. 2024-04-20T21:33:29.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:33:29.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.694+0000 7ff4a7c00640 1 -- 172.21.15.7:0/1686392381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a0074390 msgr2=0x7ff4a0074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.694+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/1686392381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a0074390 0x7ff4a0074770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff494009900 tx=0x7ff49402bcd0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.694+0000 7ff4a7c00640 1 -- 172.21.15.7:0/1686392381 shutdown_connections 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.694+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/1686392381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a0110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.695+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/1686392381 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.695+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/1686392381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a0074390 0x7ff4a0074770 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.695+0000 7ff4a7c00640 1 -- 172.21.15.7:0/1686392381 >> 172.21.15.7:0/1686392381 conn(0x7ff4a00fda40 msgr2=0x7ff4a00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.695+0000 7ff4a7c00640 1 -- 172.21.15.7:0/1686392381 shutdown_connections 2024-04-20T21:33:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.695+0000 7ff4a7c00640 1 -- 172.21.15.7:0/1686392381 wait complete. 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3862695402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:29 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3349372939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.696+0000 7ff4a7c00640 1 Processor -- start 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.696+0000 7ff4a7c00640 1 -- start start 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.696+0000 7ff4a7c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a01a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.696+0000 7ff4a7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a01a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.696+0000 7ff4a7c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a019aa90 0x7ff4a019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a7c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a0114540 con 0x7ff4a019aa90 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01143c0 con 0x7ff4a010e770 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a7c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a01146c0 con 0x7ff4a0074cb0 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a5200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a01a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a01a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a019aa90 0x7ff4a019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a5200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a01a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57098/0 (socket says 172.21.15.7:57098) 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff4a5200640 1 -- 172.21.15.7:0/2369784672 learned_addr learned my addr 172.21.15.7:0/2369784672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 -- 172.21.15.7:0/2369784672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 msgr2=0x7ff4a01a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a01a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 -- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a019aa90 msgr2=0x7ff4a019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a019aa90 0x7ff4a019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.697+0000 7ff49fe00640 1 -- 172.21.15.7:0/2369784672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a019b5b0 con 0x7ff4a010e770 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff49fe00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a01a0e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e9f0 tx=0x7ff48c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00ccc0 con 0x7ff4a010e770 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff48c010910 con 0x7ff4a010e770 2024-04-20T21:33:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c018960 con 0x7ff4a010e770 2024-04-20T21:33:29.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a019b8a0 con 0x7ff4a010e770 2024-04-20T21:33:29.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.698+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a7aa0 con 0x7ff4a010e770 2024-04-20T21:33:29.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.700+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a0069260 con 0x7ff4a010e770 2024-04-20T21:33:29.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.701+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7ff48c010ac0 con 0x7ff4a010e770 2024-04-20T21:33:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.704+0000 7ff49d600640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 0x7ff4880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.704+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff48c0d0e10 con 0x7ff4a010e770 2024-04-20T21:33:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.704+0000 7ff4a5200640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 0x7ff4880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.705+0000 7ff4a5200640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 0x7ff4880a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019bbd0 tx=0x7ff494005a20 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:29.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.707+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff48c0d4030 con 0x7ff4a010e770 2024-04-20T21:33:29.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:33:29.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:29.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3862695402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:29 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3349372939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi007:/etc/ceph/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi016:/etc/ceph/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi186:/etc/ceph/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='client.25355 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: Updating smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:29.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3862695402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:29 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3349372939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:33:29.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.961+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4a002d5d0 con 0x7ff4a010e770 2024-04-20T21:33:29.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.962+0000 7ff49d600640 1 -- 172.21.15.7:0/2369784672 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7ff48c09cc00 con 0x7ff4a010e770 2024-04-20T21:33:29.961 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:29.961 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:33:20.441301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:33:29.961 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.964+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 msgr2=0x7ff4880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.964+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 0x7ff4880a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019bbd0 tx=0x7ff494005a20 comp rx=0 tx=0).stop 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.964+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 msgr2=0x7ff4a01a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.964+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a01a0e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e9f0 tx=0x7ff48c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 shutdown_connections 2024-04-20T21:33:29.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4880a3e40 0x7ff4880a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a010e770 0x7ff4a01a0e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4a019aa90 0x7ff4a019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 --2- 172.21.15.7:0/2369784672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4a0074cb0 0x7ff4a01a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 >> 172.21.15.7:0/2369784672 conn(0x7ff4a00fda40 msgr2=0x7ff4a00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 shutdown_connections 2024-04-20T21:33:29.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:29.965+0000 7ff4a7c00640 1 -- 172.21.15.7:0/2369784672 wait complete. 2024-04-20T21:33:29.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 -- 172.21.15.7:0/2782193270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a81015a0 msgr2=0x7f96a810e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 --2- 172.21.15.7:0/2782193270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a81015a0 0x7f96a810e1d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9698009900 tx=0x7f969802bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 -- 172.21.15.7:0/2782193270 shutdown_connections 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 --2- 172.21.15.7:0/2782193270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a81015a0 0x7f96a810e1d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 --2- 172.21.15.7:0/2782193270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a810e7a0 0x7f96a8110b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 --2- 172.21.15.7:0/2782193270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a8100bf0 0x7f96a8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 -- 172.21.15.7:0/2782193270 >> 172.21.15.7:0/2782193270 conn(0x7f96a80fc820 msgr2=0x7f96a80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 -- 172.21.15.7:0/2782193270 shutdown_connections 2024-04-20T21:33:30.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.181+0000 7f96ae000640 1 -- 172.21.15.7:0/2782193270 wait complete. 2024-04-20T21:33:30.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.182+0000 7f96ae000640 1 Processor -- start 2024-04-20T21:33:30.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 -- start start 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a81015a0 0x7f96a81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a810e7a0 0x7f96a819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8114480 con 0x7f96a810e7a0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8114300 con 0x7f96a8100bf0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96ae000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8114600 con 0x7f96a81015a0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96acc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.183+0000 7f96acc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57328/0 (socket says 172.21.15.7:57328) 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 -- 172.21.15.7:0/1307234642 learned_addr learned my addr 172.21.15.7:0/1307234642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96a7e00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a81015a0 0x7f96a81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96ad600640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a810e7a0 0x7f96a819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 -- 172.21.15.7:0/1307234642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a81015a0 msgr2=0x7f96a81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a81015a0 0x7f96a81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 -- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a810e7a0 msgr2=0x7f96a819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a810e7a0 0x7f96a819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 -- 172.21.15.7:0/1307234642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a819af70 con 0x7f96a8100bf0 2024-04-20T21:33:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.184+0000 7f96acc00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ea60 tx=0x7f969c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.185+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c00cde0 con 0x7f96a8100bf0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.185+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a819b260 con 0x7f96a8100bf0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.185+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c0035f0 con 0x7f96a8100bf0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.185+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c010670 con 0x7f96a8100bf0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.185+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a81a7480 con 0x7f96a8100bf0 2024-04-20T21:33:30.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.186+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a806ad20 con 0x7f96a8100bf0 2024-04-20T21:33:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.188+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f969c0037a0 con 0x7f96a8100bf0 2024-04-20T21:33:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.189+0000 7f96a5600640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 0x7f96900a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.190+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f969c0d0d60 con 0x7f96a8100bf0 2024-04-20T21:33:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.190+0000 7f96a7e00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 0x7f96900a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.191+0000 7f96a7e00640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 0x7f96900a6370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f969802df90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.192+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f969c09d310 con 0x7f96a8100bf0 2024-04-20T21:33:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.401+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96a804fd60 con 0x7f96a8100bf0 2024-04-20T21:33:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.404+0000 7f96a5600640 1 -- 172.21.15.7:0/1307234642 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22859 (secure 0 0 0) 0x7f969c09cc10 con 0x7f96a8100bf0 2024-04-20T21:33:30.403 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:33:30.403 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:33:20.988355+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:33:30.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 msgr2=0x7f96900a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 0x7f96900a6370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9698000c00 tx=0x7f969802df90 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 msgr2=0x7f96a81a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f969c00ea60 tx=0x7f969c00ef20 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 shutdown_connections 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a810e7a0 0x7f96a819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96a81015a0 0x7f96a81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f96900a3eb0 0x7f96900a6370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 --2- 172.21.15.7:0/1307234642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a8100bf0 0x7f96a81a0630 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 >> 172.21.15.7:0/1307234642 conn(0x7f96a80fc820 msgr2=0x7f96a810fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 shutdown_connections 2024-04-20T21:33:30.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.407+0000 7f96ae000640 1 -- 172.21.15.7:0/1307234642 wait complete. 2024-04-20T21:33:30.426 DEBUG:tasks.fwd_scrub:Yielding 2024-04-20T21:33:30.426 INFO:teuthology.run_tasks:Running task exec... 2024-04-20T21:33:30.436 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:33:30.436 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:33:30.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-20T21:33:30.436 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-20T21:33:30.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:33:30.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:33:30.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:33:30.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 -- 172.21.15.7:0/3556290627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc102f40 msgr2=0x7f1acc103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/3556290627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc102f40 0x7f1acc103320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009900 tx=0x7f1abc02bcc0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 -- 172.21.15.7:0/3556290627 shutdown_connections 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/3556290627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc103860 0x7f1acc10bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/3556290627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc10c360 0x7f1acc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/3556290627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc102f40 0x7f1acc103320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 -- 172.21.15.7:0/3556290627 >> 172.21.15.7:0/3556290627 conn(0x7f1acc0fc740 msgr2=0x7f1acc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 -- 172.21.15.7:0/3556290627 shutdown_connections 2024-04-20T21:33:30.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.635+0000 7f1ad0c00640 1 -- 172.21.15.7:0/3556290627 wait complete. 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 Processor -- start 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 -- start start 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc102f40 0x7f1acc100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 0x7f1acc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112190 con 0x7f1acc102f40 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112010 con 0x7f1acc10c360 2024-04-20T21:33:30.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.636+0000 7f1ad0c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112310 con 0x7f1acc103860 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57144/0 (socket says 172.21.15.7:57144) 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 -- 172.21.15.7:0/2171300198 learned_addr learned my addr 172.21.15.7:0/2171300198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acb400640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc102f40 0x7f1acc100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acbe00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 0x7f1acc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 -- 172.21.15.7:0/2171300198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 msgr2=0x7f1acc101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 0x7f1acc101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 -- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc102f40 msgr2=0x7f1acc100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc102f40 0x7f1acc100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acaa00640 1 -- 172.21.15.7:0/2171300198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc1a6bb0 con 0x7f1acc103860 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.637+0000 7f1acbe00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 0x7f1acc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1acaa00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800e9f0 tx=0x7f1ab800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab800ccc0 con 0x7f1acc103860 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc1a6ea0 con 0x7f1acc103860 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab800ce60 con 0x7f1acc103860 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8010690 con 0x7f1acc103860 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.638+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a7310 con 0x7f1acc103860 2024-04-20T21:33:30.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.639+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc068800 con 0x7f1acc103860 2024-04-20T21:33:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f1ab80046c0 con 0x7f1acc103860 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 -- 172.21.15.7:0/3207585887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c410e620 msgr2=0x7f00c4110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 --2- 172.21.15.7:0/3207585887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c410e620 0x7f00c4110a10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009ca0 tx=0x7f00bc02c190 comp rx=0 tx=0).stop 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 -- 172.21.15.7:0/3207585887 shutdown_connections 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 --2- 172.21.15.7:0/3207585887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c410e620 0x7f00c4110a10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 --2- 172.21.15.7:0/3207585887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c4074c30 0x7f00c410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 --2- 172.21.15.7:0/3207585887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 0x7f00c40746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 -- 172.21.15.7:0/3207585887 >> 172.21.15.7:0/3207585887 conn(0x7f00c40fd8b0 msgr2=0x7f00c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 -- 172.21.15.7:0/3207585887 shutdown_connections 2024-04-20T21:33:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.644+0000 7f00c8e00640 1 -- 172.21.15.7:0/3207585887 wait complete. 2024-04-20T21:33:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.645+0000 7f1ac3e00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 0x7f1ab00a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.645+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ab80d0e70 con 0x7f1acc103860 2024-04-20T21:33:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.645+0000 7f1acb400640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 0x7f1ab00a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.646+0000 7f00c8e00640 1 Processor -- start 2024-04-20T21:33:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.646+0000 7f00c8e00640 1 -- start start 2024-04-20T21:33:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.646+0000 7f1acb400640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 0x7f1ab00a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005ab0 tx=0x7f1abc005a20 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 0x7f00c41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c410e620 0x7f00c419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4114300 con 0x7f00c410e620 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4114180 con 0x7f00c4074310 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c8e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f00c4114480 con 0x7f00c4074c30 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 0x7f00c41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c410e620 0x7f00c419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57166/0 (socket says 172.21.15.7:57166) 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 -- 172.21.15.7:0/1691667009 learned_addr learned my addr 172.21.15.7:0/1691667009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 -- 172.21.15.7:0/1691667009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 msgr2=0x7f00c41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 0x7f00c41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 -- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c410e620 msgr2=0x7f00c419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c410e620 0x7f00c419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.647+0000 7f00c2a00640 1 -- 172.21.15.7:0/1691667009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c419af40 con 0x7f00c4074c30 2024-04-20T21:33:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.648+0000 7f00c2a00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e9f0 tx=0x7f00b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f00bbe00640 1 -- 172.21.15.7:0/1691667009 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000cd80 con 0x7f00c4074c30 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f00bbe00640 1 -- 172.21.15.7:0/1691667009 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b0010910 con 0x7f00c4074c30 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c419b230 con 0x7f00c4074c30 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f00bbe00640 1 -- 172.21.15.7:0/1691667009 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0018990 con 0x7f00c4074c30 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c41a74f0 con 0x7f00c4074c30 2024-04-20T21:33:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.649+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ab809d2b0 con 0x7f1acc103860 2024-04-20T21:33:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.652+0000 7f00bbe00640 1 -- 172.21.15.7:0/1691667009 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f00b0010ac0 con 0x7f00c4074c30 2024-04-20T21:33:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.653+0000 7f00cd18b740 1 Processor -- start 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.653+0000 7f00cd18b740 1 -- start start 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.653+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a5844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.653+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00cd18b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a4e8d20 con 0x55bc8a5840f0 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00cd18b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a48f3e0 con 0x55bc8a58d5f0 2024-04-20T21:33:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00cd18b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a4e4ce0 con 0x55bc8a584a10 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57168/0 (socket says 172.21.15.7:57168) 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 -- 172.21.15.7:0/2948506041 learned_addr learned my addr 172.21.15.7:0/2948506041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3e00640 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a5844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 -- 172.21.15.7:0/2948506041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 msgr2=0x55bc8a58f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 -- 172.21.15.7:0/2948506041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 msgr2=0x55bc8a5844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a5844d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c3400640 1 -- 172.21.15.7:0/2948506041 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8a4ae980 con 0x55bc8a584a10 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.654+0000 7f00c2a00640 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00bbe00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 0x7f00a80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00c3400640 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009cf0 tx=0x7f00b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64be03a25acaa268 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00bbe00640 1 -- 172.21.15.7:0/1691667009 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f00b00d0ca0 con 0x7f00c4074c30 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00c3e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 0x7f00a80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00b8c00640 1 -- 172.21.15.7:0/2948506041 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4004230 con 0x55bc8a584a10 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00b8c00640 1 -- 172.21.15.7:0/2948506041 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40043d0 con 0x55bc8a584a10 2024-04-20T21:33:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.655+0000 7f00b8c00640 1 -- 172.21.15.7:0/2948506041 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4019960 con 0x55bc8a584a10 2024-04-20T21:33:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.656+0000 7f00c3e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 0x7f00a80a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005e30 tx=0x7f00bc005d80 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.656+0000 7f00cd18b740 1 -- 172.21.15.7:0/2948506041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 msgr2=0x55bc8a58d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.656+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009cf0 tx=0x7f00b4011810 comp rx=0 tx=0).stop 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.656+0000 7f00cd18b740 1 -- 172.21.15.7:0/2948506041 shutdown_connections 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a58d0b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2948506041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a5844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 -- 172.21.15.7:0/2948506041 >> 172.21.15.7:0/2948506041 conn(0x55bc8a57e820 msgr2=0x55bc8a57ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 -- 172.21.15.7:0/2948506041 shutdown_connections 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 -- 172.21.15.7:0/2948506041 wait complete. 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.657+0000 7f00cd18b740 1 -- start start 2024-04-20T21:33:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.658+0000 7f00cd18b740 1 Processor -- start 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.658+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.658+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a58abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00cd18b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a5903e0 con 0x55bc8a584a10 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00cd18b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a4a27a0 con 0x55bc8a58d5f0 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00cd18b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bc8a590560 con 0x55bc8a5840f0 2024-04-20T21:33:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00cd18b740 10 client.0 _ll_register_callbacks cb 0x55bc8a4a4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a58abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57172/0 (socket says 172.21.15.7:57172) 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57388/0 (socket says 172.21.15.7:57388) 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 -- 172.21.15.7:0/1272590672 learned_addr learned my addr 172.21.15.7:0/1272590672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 -- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 msgr2=0x55bc8a58b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 -- 172.21.15.7:0/1272590672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 msgr2=0x55bc8a58abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a58abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.659+0000 7f00c3400640 1 -- 172.21.15.7:0/1272590672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8a2c37e0 con 0x55bc8a5840f0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00c3400640 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00b40052a0 tx=0x7f00b40045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d76e365fabca603 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00ad600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4027070 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f00b40037f0 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b40043d0 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f00b4033050 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00cd18b740 10 client.25405 fetch_fsmap learned FSMap version 24 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.660+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc8a323f70 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40043d0 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f00b4004ba0 con 0x55bc8a5840f0 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 10 client.25405 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 20 client.25405 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a59d910 0x55bc8a59fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 4 client.25405 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.661+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc8a4f6740 con 0x55bc8a59d910 2024-04-20T21:33:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.662+0000 7f00c2a00640 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a59d910 0x55bc8a59fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.662+0000 7f00c2a00640 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a59d910 0x55bc8a59fd00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbb62e467c5e1a6 server_cookie=54b72c29388d5dfe in_seq=0 out_seq=0 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.662+0000 7f00ad600640 10 client.25405 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00ad600640 1 -- 172.21.15.7:0/1272590672 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc8a4f6740 con 0x55bc8a59d910 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00ad600640 10 client.25405 handle_command_reply: tid=0 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 client.25405 shutdown 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 msgr2=0x55bc8a58a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00b40052a0 tx=0x7f00b40045f0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 shutdown_connections 2024-04-20T21:33:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a58b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a59d910 0x55bc8a59fd00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a58abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 --2- 172.21.15.7:0/1272590672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a58a670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00ad600640 0 client.25405 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.663+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 >> 172.21.15.7:0/1272590672 conn(0x55bc8a57e820 msgr2=0x55bc8a580570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.664+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 shutdown_connections 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.664+0000 7f00cd18b740 1 -- 172.21.15.7:0/1272590672 wait complete. 2024-04-20T21:33:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.664+0000 7f00cd18b740 20 client.25405 trim_cache size 0 max 16384 2024-04-20T21:33:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-20T21:33:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/556933149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2369784672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:30 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.705+0000 7f00cd18b740 1 Processor -- start 2024-04-20T21:33:30.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.705+0000 7f00cd18b740 1 -- start start 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a589f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a5992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41b2570 con 0x55bc8a5840f0 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41afe30 con 0x55bc8a58d5f0 2024-04-20T21:33:30.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00cd18b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41adc80 con 0x55bc8a584a10 2024-04-20T21:33:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a589f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a5992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57390/0 (socket says 172.21.15.7:57390) 2024-04-20T21:33:30.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a5992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57174/0 (socket says 172.21.15.7:57174) 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.706+0000 7f00c3e00640 1 -- 172.21.15.7:0/2540259761 learned_addr learned my addr 172.21.15.7:0/2540259761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 -- 172.21.15.7:0/2540259761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 msgr2=0x55bc8a5992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a5992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 -- 172.21.15.7:0/2540259761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 msgr2=0x55bc8a589f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a589f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 -- 172.21.15.7:0/2540259761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc8a57e080 con 0x55bc8a58d5f0 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.707+0000 7f00c3e00640 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009ca0 tx=0x7f00bc041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459584ffa0487e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.708+0000 7f00afe00640 1 -- 172.21.15.7:0/2540259761 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc039040 con 0x55bc8a58d5f0 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.708+0000 7f00afe00640 1 -- 172.21.15.7:0/2540259761 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00bc0046a0 con 0x55bc8a58d5f0 2024-04-20T21:33:30.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.708+0000 7f00afe00640 1 -- 172.21.15.7:0/2540259761 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc04e460 con 0x55bc8a58d5f0 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.708+0000 7f00cd18b740 1 -- 172.21.15.7:0/2540259761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 msgr2=0x55bc8a5a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.708+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009ca0 tx=0x7f00bc041000 comp rx=0 tx=0).stop 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 -- 172.21.15.7:0/2540259761 shutdown_connections 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a5a5470 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a584a10 0x55bc8a5992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 --2- 172.21.15.7:0/2540259761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a5840f0 0x55bc8a589f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 -- 172.21.15.7:0/2540259761 >> 172.21.15.7:0/2540259761 conn(0x55bc8a57e820 msgr2=0x55bc8a5802e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 -- 172.21.15.7:0/2540259761 shutdown_connections 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 -- 172.21.15.7:0/2540259761 wait complete. 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 -- start start 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.709+0000 7f00cd18b740 1 Processor -- start 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a587c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a60b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41ae2f0 con 0x55bc8a584a10 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41af760 con 0x55bc8a58d5f0 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f00c41ae9c0 con 0x55bc8a5840f0 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00cd18b740 10 client.0 _ll_register_callbacks cb 0x7f00b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57176/0 (socket says 172.21.15.7:57176) 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 -- 172.21.15.7:0/3109334233 learned_addr learned my addr 172.21.15.7:0/3109334233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c2a00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a587c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3e00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a60b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 -- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 msgr2=0x55bc8a60b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a60b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 -- 172.21.15.7:0/3109334233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 msgr2=0x55bc8a587c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.710+0000 7f00c3400640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a587c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00c3400640 1 -- 172.21.15.7:0/3109334233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc8a584840 con 0x55bc8a5840f0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00c3e00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a60b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00c2a00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a587c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00c3400640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00b40058c0 tx=0x7f00b4009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb6b0cbd747974b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00ad600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b403c060 con 0x55bc8a5840f0 2024-04-20T21:33:30.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f00b4019a00 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.711+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00bc0046a0 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f00b4027070 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 10 client.25411 fetch_fsmap learned FSMap version 24 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bc8a1bb170 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00bc0046a0 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f00b4019c40 con 0x55bc8a5840f0 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 10 client.25411 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 20 client.25411 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a60cf20 0x55bc8a60f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 4 client.25411 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.712+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc8a4f05b0 con 0x55bc8a60cf20 2024-04-20T21:33:30.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.713+0000 7f00c2a00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a60cf20 0x55bc8a60f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.713+0000 7f00ad600640 10 client.25411 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.713+0000 7f00c2a00640 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a60cf20 0x55bc8a60f340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=600cbeb64947ed60 server_cookie=c40d1c416142c760 in_seq=0 out_seq=0 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.714+0000 7f00ad600640 1 -- 172.21.15.7:0/3109334233 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc8a4f05b0 con 0x55bc8a60cf20 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.714+0000 7f00ad600640 10 client.25411 handle_command_reply: tid=0 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.714+0000 7f00cd18b740 1 client.25411 shutdown 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 msgr2=0x55bc8a5876f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f00b40058c0 tx=0x7f00b4009b70 comp rx=0 tx=0).stop 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 shutdown_connections 2024-04-20T21:33:30.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bc8a58d5f0 0x55bc8a60b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bc8a60cf20 0x55bc8a60f340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bc8a584a10 0x55bc8a587c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 --2- 172.21.15.7:0/3109334233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bc8a5840f0 0x55bc8a5876f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00ad600640 0 client.25411 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 >> 172.21.15.7:0/3109334233 conn(0x55bc8a57e820 msgr2=0x55bc8a58a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 shutdown_connections 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 1 -- 172.21.15.7:0/3109334233 wait complete. 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.715+0000 7f00cd18b740 20 client.25411 trim_cache size 0 max 16384 2024-04-20T21:33:30.717 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:33:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:33:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "e51bb57b-212e-4d1b-b6b3-cc8002ab6d55", 2024-04-20T21:33:30.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:33:30.718 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 msgr2=0x7f00a80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 0x7f00a80a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005e30 tx=0x7f00bc005d80 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 msgr2=0x7f00c41a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e9f0 tx=0x7f00b000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 shutdown_connections 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c410e620 0x7f00c419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f00c4074c30 0x7f00c41a0ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f00a80a3ec0 0x7f00a80a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 --2- 172.21.15.7:0/1691667009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00c4074310 0x7f00c41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.716+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 >> 172.21.15.7:0/1691667009 conn(0x7f00c40fd8b0 msgr2=0x7f00c40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.717+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 shutdown_connections 2024-04-20T21:33:30.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.717+0000 7f00c8e00640 1 -- 172.21.15.7:0/1691667009 wait complete. 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/556933149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2369784672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:30 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.730 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: Updating smithi007:/etc/ceph/ceph.client.admin.keyring 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 23 op/s 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: Updating smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/556933149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2369784672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:30.731 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:30 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:33:30.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:33:30.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:33:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.920+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1acc0623a0 con 0x7f1ab00a3e50 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 -- 172.21.15.7:0/3698446552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3101092b0 msgr2=0x7fc310110e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 --2- 172.21.15.7:0/3698446552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3101092b0 0x7fc310110e80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc308009be0 tx=0x7fc30802bef0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 -- 172.21.15.7:0/3698446552 shutdown_connections 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 --2- 172.21.15.7:0/3698446552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc310104b90 0x7fc310108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 --2- 172.21.15.7:0/3698446552 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc3101092b0 0x7fc310110e80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 --2- 172.21.15.7:0/3698446552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 -- 172.21.15.7:0/3698446552 >> 172.21.15.7:0/3698446552 conn(0x7fc3100ffa40 msgr2=0x7fc310101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 -- 172.21.15.7:0/3698446552 shutdown_connections 2024-04-20T21:33:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.930+0000 7fc315400640 1 -- 172.21.15.7:0/3698446552 wait complete. 2024-04-20T21:33:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.931+0000 7fc315400640 1 Processor -- start 2024-04-20T21:33:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 -- start start 2024-04-20T21:33:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 0x7fc3101a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 0x7fc31019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101143a0 con 0x7fc3101041e0 2024-04-20T21:33:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc310114220 con 0x7fc3101092b0 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc315400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc310114520 con 0x7fc31019a620 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.932+0000 7fc30f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 0x7fc3101a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc30f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 0x7fc3101a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57414/0 (socket says 172.21.15.7:57414) 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc30f400640 1 -- 172.21.15.7:0/1347118523 learned_addr learned my addr 172.21.15.7:0/1347118523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 0x7fc31019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc30fe00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 -- 172.21.15.7:0/1347118523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 msgr2=0x7fc3101a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 0x7fc3101a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 -- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 msgr2=0x7fc3101a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc314a00640 1 -- 172.21.15.7:0/1347118523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31019b140 con 0x7fc31019a620 2024-04-20T21:33:30.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.933+0000 7fc30fe00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc314a00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 0x7fc31019aad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc30802c430 tx=0x7fc30802cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc30cc00640 1 -- 172.21.15.7:0/1347118523 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308043020 con 0x7fc31019a620 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc30cc00640 1 -- 172.21.15.7:0/1347118523 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc308003e50 con 0x7fc31019a620 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc30cc00640 1 -- 172.21.15.7:0/1347118523 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30803e6b0 con 0x7fc31019a620 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31019b3d0 con 0x7fc31019a620 2024-04-20T21:33:30.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.934+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3101a7670 con 0x7fc31019a620 2024-04-20T21:33:30.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.939+0000 7fc319838740 1 Processor -- start 2024-04-20T21:33:30.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.940+0000 7fc30cc00640 1 -- 172.21.15.7:0/1347118523 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fc308046020 con 0x7fc31019a620 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.940+0000 7fc319838740 1 -- start start 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b27f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b33480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d110a85150 con 0x55d110b27b90 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d110a2b370 con 0x55d110b284b0 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc319838740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d110a81110 con 0x55d110b31090 2024-04-20T21:33:30.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57422/0 (socket says 172.21.15.7:57422) 2024-04-20T21:33:30.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.941+0000 7fc30fe00640 1 -- 172.21.15.7:0/2735400875 learned_addr learned my addr 172.21.15.7:0/2735400875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30f400640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b33480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30f400640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b27f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30fe00640 1 -- 172.21.15.7:0/2735400875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 msgr2=0x55d110b33480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b33480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30fe00640 1 -- 172.21.15.7:0/2735400875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 msgr2=0x55d110b27f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b27f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30fe00640 1 -- 172.21.15.7:0/2735400875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1108617d0 con 0x55d110b284b0 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.942+0000 7fc30f400640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b27f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc30000bcc0 tx=0x7fc30000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d0040dfc3472937 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc305200640 1 -- 172.21.15.7:0/2735400875 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300005020 con 0x55d110b284b0 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc30cc00640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 0x7fc2f40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc305200640 1 -- 172.21.15.7:0/2735400875 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3000047a0 con 0x55d110b284b0 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc305200640 1 -- 172.21.15.7:0/2735400875 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300011740 con 0x55d110b284b0 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.943+0000 7fc30cc00640 1 -- 172.21.15.7:0/1347118523 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc3080f2f80 con 0x7fc31019a620 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc30f400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 0x7fc2f40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 -- 172.21.15.7:0/2735400875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 msgr2=0x55d110b30b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc30000bcc0 tx=0x7fc30000f730 comp rx=0 tx=0).stop 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 -- 172.21.15.7:0/2735400875 shutdown_connections 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b284b0 0x55d110b30b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b33480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 --2- 172.21.15.7:0/2735400875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b27f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 -- 172.21.15.7:0/2735400875 >> 172.21.15.7:0/2735400875 conn(0x55d110b222c0 msgr2=0x55d110b226a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 -- 172.21.15.7:0/2735400875 shutdown_connections 2024-04-20T21:33:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc319838740 1 -- 172.21.15.7:0/2735400875 wait complete. 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.944+0000 7fc30f400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 0x7fc2f40a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55d110b23450 tx=0x7fc2fc009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.945+0000 7fc319838740 1 -- start start 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.945+0000 7fc319838740 1 Processor -- start 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b1f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d110b19780 con 0x55d110b27b90 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d110a3e740 con 0x55d110b31090 2024-04-20T21:33:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d110b19900 con 0x55d110b284b0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc319838740 10 client.0 _ll_register_callbacks cb 0x55d110a39b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc30f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc30f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57204/0 (socket says 172.21.15.7:57204) 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc30f400640 1 -- 172.21.15.7:0/3844594259 learned_addr learned my addr 172.21.15.7:0/3844594259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b1f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.946+0000 7fc314a00640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 -- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 msgr2=0x55d110b2fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 -- 172.21.15.7:0/3844594259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 msgr2=0x55d110b1f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b1f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 -- 172.21.15.7:0/3844594259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1107973f0 con 0x55d110b284b0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc30f400640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011c00 tx=0x7fc2fc009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=125dae0bbf24c7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc2f9600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc010d50 con 0x55d110b284b0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc2fc0203f0 con 0x55d110b284b0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.947+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3000047a0 con 0x55d110b284b0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.948+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc0205f0 con 0x55d110b284b0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.948+0000 7fc319838740 10 client.25423 fetch_fsmap learned FSMap version 24 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.948+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1107570f0 con 0x55d110b284b0 2024-04-20T21:33:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.948+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3000047a0 con 0x55d110b284b0 2024-04-20T21:33:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fc2fc031a30 con 0x55d110b284b0 2024-04-20T21:33:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 10 client.25423 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:33:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 20 client.25423 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110b49b00 0x55d110b4bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 4 client.25423 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.949+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d110a92b70 con 0x55d110b49b00 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.950+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110b49b00 0x55d110b4bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.950+0000 7fc2f9600640 10 client.25423 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.950+0000 7fc30fe00640 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110b49b00 0x55d110b4bf30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62151d0527f5e10 server_cookie=88492803d4a1b2d9 in_seq=0 out_seq=0 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.951+0000 7fc2f9600640 1 -- 172.21.15.7:0/3844594259 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d110a92b70 con 0x55d110b49b00 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.951+0000 7fc2f9600640 10 client.25423 handle_command_reply: tid=0 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.951+0000 7fc319838740 1 client.25423 shutdown 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 msgr2=0x55d110b1f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc011c00 tx=0x7fc2fc009560 comp rx=0 tx=0).stop 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 shutdown_connections 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110b49b00 0x55d110b4bf30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b284b0 0x55d110b1f950 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 --2- 172.21.15.7:0/3844594259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b27b90 0x55d110b1f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 >> 172.21.15.7:0/3844594259 conn(0x55d110b222c0 msgr2=0x55d110b24010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc2f9600640 0 client.25423 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 shutdown_connections 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 1 -- 172.21.15.7:0/3844594259 wait complete. 2024-04-20T21:33:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.952+0000 7fc319838740 20 client.25423 trim_cache size 0 max 16384 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 Processor -- start 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 -- start start 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b27b90 0x55d110b1e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101ae0e0 con 0x55d110b284b0 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101ab9a0 con 0x55d110b27b90 2024-04-20T21:33:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.978+0000 7fc319838740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101a9810 con 0x55d110b31090 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57218/0 (socket says 172.21.15.7:57218) 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 -- 172.21.15.7:0/1055773339 learned_addr learned my addr 172.21.15.7:0/1055773339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc30fe00640 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b27b90 0x55d110b1e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc30f400640 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 -- 172.21.15.7:0/1055773339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b27b90 msgr2=0x55d110b1e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b27b90 0x55d110b1e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 -- 172.21.15.7:0/1055773339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 msgr2=0x55d110b2fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.979+0000 7fc314a00640 1 -- 172.21.15.7:0/1055773339 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d110b1d770 con 0x55d110b31090 2024-04-20T21:33:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.980+0000 7fc314a00640 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3080f0d20 tx=0x7fc3080bec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd7174f671dde7f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.980+0000 7fc2fbe00640 1 -- 172.21.15.7:0/1055773339 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004840 con 0x55d110b31090 2024-04-20T21:33:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.980+0000 7fc2fbe00640 1 -- 172.21.15.7:0/1055773339 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080bf530 con 0x55d110b31090 2024-04-20T21:33:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.980+0000 7fc2fbe00640 1 -- 172.21.15.7:0/1055773339 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080d8270 con 0x55d110b31090 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- 172.21.15.7:0/1055773339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 msgr2=0x55d110b51800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc3080f0d20 tx=0x7fc3080bec00 comp rx=0 tx=0).stop 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- 172.21.15.7:0/1055773339 shutdown_connections 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b31090 0x55d110b51800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 --2- 172.21.15.7:0/1055773339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b27b90 0x55d110b1e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- 172.21.15.7:0/1055773339 >> 172.21.15.7:0/1055773339 conn(0x55d110b222c0 msgr2=0x55d110b1daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- 172.21.15.7:0/1055773339 shutdown_connections 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- 172.21.15.7:0/1055773339 wait complete. 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.981+0000 7fc319838740 1 -- start start 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 Processor -- start 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101a9e60 con 0x55d110b284b0 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101ab2d0 con 0x55d110b31090 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101aa530 con 0x55d110b27b90 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc319838740 10 client.0 _ll_register_callbacks cb 0x7fc2fc009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:33:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc30fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:57234/0 (socket says 172.21.15.7:57234) 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.982+0000 7fc30fe00640 1 -- 172.21.15.7:0/2915308874 learned_addr learned my addr 172.21.15.7:0/2915308874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc314a00640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30f400640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30fe00640 1 -- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 msgr2=0x55d110b2bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30fe00640 1 -- 172.21.15.7:0/2915308874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 msgr2=0x55d110b2b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc30fe00640 1 -- 172.21.15.7:0/2915308874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d110baf270 con 0x55d110b27b90 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.983+0000 7fc314a00640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc30fe00640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc30000fc30 tx=0x7fc30000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7c44beff1d6f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc2f9600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3000035f0 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc300003790 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3080bf530 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc300003990 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc319838740 10 client.25429 fetch_fsmap learned FSMap version 24 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d110bb0520 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.984+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3080bf530 con 0x55d110b27b90 2024-04-20T21:33:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fc300027440 con 0x55d110b27b90 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 10 client.25429 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 20 client.25429 populate_metadata read hostname 'smithi007' 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110bb0ae0 0x55d110bb2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 4 client.25429 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d110b15170 con 0x55d110bb0ae0 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.985+0000 7fc30f400640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110bb0ae0 0x55d110bb2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc30f400640 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110bb0ae0 0x55d110bb2f00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a794de069ea83a77 server_cookie=50e31093131f5d0f in_seq=0 out_seq=0 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc2f9600640 10 client.25429 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc2f9600640 1 -- 172.21.15.7:0/2915308874 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d110b15170 con 0x55d110bb0ae0 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc2f9600640 10 client.25429 handle_command_reply: tid=0 2024-04-20T21:33:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 client.25429 shutdown 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 msgr2=0x55d110b2b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc30000fc30 tx=0x7fc30000ba10 comp rx=0 tx=0).stop 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 shutdown_connections 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d110b31090 0x55d110b2bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d110bb0ae0 0x55d110bb2f00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d110b284b0 0x55d110b2b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc2f9600640 0 client.25429 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 --2- 172.21.15.7:0/2915308874 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d110b27b90 0x55d110b2b1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.986+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 >> 172.21.15.7:0/2915308874 conn(0x55d110b222c0 msgr2=0x55d110b18280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.987+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 shutdown_connections 2024-04-20T21:33:30.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.987+0000 7fc319838740 1 -- 172.21.15.7:0/2915308874 wait complete. 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.987+0000 7fc319838740 20 client.25429 trim_cache size 0 max 16384 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "e51bb57b-212e-4d1b-b6b3-cc8002ab6d55": { 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "e51bb57b-212e-4d1b-b6b3-cc8002ab6d55", 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:33:30.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:33:30.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:33:30.991 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:33:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 msgr2=0x7fc2f40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 0x7fc2f40a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55d110b23450 tx=0x7fc2fc009260 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 msgr2=0x7fc31019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 0x7fc31019aad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc30802c430 tx=0x7fc30802cbc0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 shutdown_connections 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc2f40a3ec0 0x7fc2f40a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3101092b0 0x7fc3101a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc31019a620 0x7fc31019aad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 --2- 172.21.15.7:0/1347118523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3101041e0 0x7fc3101a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.988+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 >> 172.21.15.7:0/1347118523 conn(0x7fc3100ffa40 msgr2=0x7fc310100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.989+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 shutdown_connections 2024-04-20T21:33:30.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:30.989+0000 7fc315400640 1 -- 172.21.15.7:0/1347118523 wait complete. 2024-04-20T21:33:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.001+0000 7f1ac3e00640 1 -- 172.21.15.7:0/2171300198 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1acc0623a0 con 0x7f1ab00a3e50 2024-04-20T21:33:30.999 INFO:teuthology.orchestra.run.smithi007.stdout:Schedule set for path / 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 msgr2=0x7f1ab00a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 0x7f1ab00a6310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1abc005ab0 tx=0x7f1abc005a20 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 msgr2=0x7f1acc1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800e9f0 tx=0x7f1ab800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 shutdown_connections 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ab00a3e50 0x7f1ab00a6310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1acc10c360 0x7f1acc101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1acc103860 0x7f1acc1010d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 --2- 172.21.15.7:0/2171300198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1acc102f40 0x7f1acc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 >> 172.21.15.7:0/2171300198 conn(0x7f1acc0fc740 msgr2=0x7f1acc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 shutdown_connections 2024-04-20T21:33:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.003+0000 7f1ad0c00640 1 -- 172.21.15.7:0/2171300198 wait complete. 2024-04-20T21:33:31.007 INFO:tasks.cephfs.filesystem:scrub status for tag:e51bb57b-212e-4d1b-b6b3-cc8002ab6d55 - {'path': '/', 'tag': 'e51bb57b-212e-4d1b-b6b3-cc8002ab6d55', 'options': 'recursive,force'} 2024-04-20T21:33:31.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 -- 172.21.15.7:0/707039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c104d80 msgr2=0x7ff61c108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 --2- 172.21.15.7:0/707039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c104d80 0x7ff61c108dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff60c009900 tx=0x7ff60c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 -- 172.21.15.7:0/707039747 shutdown_connections 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 --2- 172.21.15.7:0/707039747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 0x7ff61c111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 --2- 172.21.15.7:0/707039747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c104d80 0x7ff61c108dc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 --2- 172.21.15.7:0/707039747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c1043d0 0x7ff61c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 -- 172.21.15.7:0/707039747 >> 172.21.15.7:0/707039747 conn(0x7ff61c0ffc50 msgr2=0x7ff61c102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 -- 172.21.15.7:0/707039747 shutdown_connections 2024-04-20T21:33:31.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.249+0000 7ff621400640 1 -- 172.21.15.7:0/707039747 wait complete. 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 Processor -- start 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 -- start start 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 0x7ff61c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 0x7ff61c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114580 con 0x7ff61c1043d0 2024-04-20T21:33:31.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114400 con 0x7ff61c1094a0 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.250+0000 7ff621400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114700 con 0x7ff61c19a990 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57734/0 (socket says 172.21.15.7:57734) 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 0x7ff61c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61a800640 1 -- 172.21.15.7:0/3323000153 learned_addr learned my addr 172.21.15.7:0/3323000153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff619e00640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 0x7ff61c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 -- 172.21.15.7:0/3323000153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 msgr2=0x7ff61c1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 0x7ff61c1a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 -- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 msgr2=0x7ff61c1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61b200640 1 -- 172.21.15.7:0/3323000153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c19b4b0 con 0x7ff61c19a990 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.251+0000 7ff61a800640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:33:31.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff61b200640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 0x7ff61c19ae40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e970 tx=0x7ff60400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60400ccc0 con 0x7ff61c19a990 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c19b7a0 con 0x7ff61c19a990 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6040035f0 con 0x7ff61c19a990 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60400ce60 con 0x7ff61c19a990 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.252+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a79a0 con 0x7ff61c19a990 2024-04-20T21:33:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.253+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c050540 con 0x7ff61c19a990 2024-04-20T21:33:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.255+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7ff6040037a0 con 0x7ff61c19a990 2024-04-20T21:33:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.257+0000 7ff613400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 0x7ff6000a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.257+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff6040d0c80 con 0x7ff61c19a990 2024-04-20T21:33:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.257+0000 7ff61a800640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 0x7ff6000a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.258+0000 7ff61a800640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 0x7ff6000a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff61c103390 tx=0x7ff614009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.259+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff60409d1d0 con 0x7ff61c19a990 2024-04-20T21:33:31.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.526+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff61c104d80 con 0x7ff6000a3e30 2024-04-20T21:33:31.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.680+0000 7ff613400640 1 -- 172.21.15.7:0/3323000153 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff61c104d80 con 0x7ff6000a3e30 2024-04-20T21:33:31.679 INFO:teuthology.orchestra.run.smithi007.stdout:Retention added to path / 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 msgr2=0x7ff6000a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 0x7ff6000a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff61c103390 tx=0x7ff614009500 comp rx=0 tx=0).stop 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 msgr2=0x7ff61c19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 0x7ff61c19ae40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff60400e970 tx=0x7ff60400ee30 comp rx=0 tx=0).stop 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 shutdown_connections 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6000a3e30 0x7ff6000a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61c1094a0 0x7ff61c1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c19a990 0x7ff61c19ae40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 --2- 172.21.15.7:0/3323000153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff61c1043d0 0x7ff61c1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 >> 172.21.15.7:0/3323000153 conn(0x7ff61c0ffc50 msgr2=0x7ff61c101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 shutdown_connections 2024-04-20T21:33:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.684+0000 7ff621400640 1 -- 172.21.15.7:0/3323000153 wait complete. 2024-04-20T21:33:31.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1307234642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:31.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:31 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1307234642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:31.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.700 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:31.701 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:31 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1307234642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:33:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:31 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 -- 172.21.15.7:0/2509501756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 msgr2=0x7fb8001023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/2509501756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001023b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009900 tx=0x7fb7f002bc40 comp rx=0 tx=0).stop 2024-04-20T21:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 -- 172.21.15.7:0/2509501756 shutdown_connections 2024-04-20T21:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/2509501756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb800102980 0x7fb8001069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/2509501756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb8001070a0 0x7fb80010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/2509501756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001023b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 -- 172.21.15.7:0/2509501756 >> 172.21.15.7:0/2509501756 conn(0x7fb8000fd890 msgr2=0x7fb8000ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 -- 172.21.15.7:0/2509501756 shutdown_connections 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.906+0000 7fb7ffe00640 1 -- 172.21.15.7:0/2509501756 wait complete. 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.908+0000 7fb7ffe00640 1 Processor -- start 2024-04-20T21:33:31.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.908+0000 7fb7ffe00640 1 -- start start 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.908+0000 7fb7ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.908+0000 7fb7ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8001070a0 0x7fb800198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.908+0000 7fb7ffe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb80019cf60 0x7fb80019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7ffe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb800112180 con 0x7fb800101fd0 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7ffe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb800112000 con 0x7fb8001070a0 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7ffe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb800112300 con 0x7fb80019cf60 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57756/0 (socket says 172.21.15.7:57756) 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 -- 172.21.15.7:0/1196796563 learned_addr learned my addr 172.21.15.7:0/1196796563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fe000640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8001070a0 0x7fb800198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7ff400640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb80019cf60 0x7fb80019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 -- 172.21.15.7:0/1196796563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb80019cf60 msgr2=0x7fb80019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb80019cf60 0x7fb80019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 -- 172.21.15.7:0/1196796563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8001070a0 msgr2=0x7fb800198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8001070a0 0x7fb800198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.909+0000 7fb7fea00640 1 -- 172.21.15.7:0/1196796563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80019da50 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb7fea00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005ab0 tx=0x7fb7f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0041070 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f002cca0 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80019dce0 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0003d20 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.910+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001a54a0 con 0x7fb800101fd0 2024-04-20T21:33:31.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.912+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8001999f0 con 0x7fb800101fd0 2024-04-20T21:33:31.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.914+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7fb7f0044070 con 0x7fb800101fd0 2024-04-20T21:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.915+0000 7fb804a00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 0x7fb7e40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.915+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb7f00f2300 con 0x7fb800101fd0 2024-04-20T21:33:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.915+0000 7fb7fe000640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 0x7fb7e40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.916+0000 7fb7fe000640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 0x7fb7e40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb800108650 tx=0x7fb7ec005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:31.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:31.920+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb7f00be850 con 0x7fb800101fd0 2024-04-20T21:33:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.154+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb800102980 con 0x7fb7e40a3e20 2024-04-20T21:33:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.173+0000 7fb804a00640 1 -- 172.21.15.7:0/1196796563 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb800102980 con 0x7fb7e40a3e20 2024-04-20T21:33:32.174 INFO:teuthology.orchestra.run.smithi007.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T21:33:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-20T21:33:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.175+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 msgr2=0x7fb7e40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.175+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 0x7fb7e40a62e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb800108650 tx=0x7fb7ec005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.175+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 msgr2=0x7fb8001986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.175+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005ab0 tx=0x7fb7f00042f0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 shutdown_connections 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb7e40a3e20 0x7fb7e40a62e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8001070a0 0x7fb800198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb80019cf60 0x7fb80019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 --2- 172.21.15.7:0/1196796563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb800101fd0 0x7fb8001986a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 >> 172.21.15.7:0/1196796563 conn(0x7fb8000fd890 msgr2=0x7fb8000ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 shutdown_connections 2024-04-20T21:33:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.176+0000 7fb7ffe00640 1 -- 172.21.15.7:0/1196796563 wait complete. 2024-04-20T21:33:32.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T21:33:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 -- 172.21.15.7:0/1301570720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408106120 msgr2=0x7f34081065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 --2- 172.21.15.7:0/1301570720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408106120 0x7f34081065b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3400009be0 tx=0x7f340002bfd0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 -- 172.21.15.7:0/1301570720 shutdown_connections 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 --2- 172.21.15.7:0/1301570720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408101640 0x7f3408105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 --2- 172.21.15.7:0/1301570720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408106120 0x7f34081065b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 --2- 172.21.15.7:0/1301570720 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 0x7f3408101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 -- 172.21.15.7:0/1301570720 >> 172.21.15.7:0/1301570720 conn(0x7f34080fc770 msgr2=0x7f34080feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 -- 172.21.15.7:0/1301570720 shutdown_connections 2024-04-20T21:33:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.410+0000 7f3407200640 1 -- 172.21.15.7:0/1301570720 wait complete. 2024-04-20T21:33:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.411+0000 7f3407200640 1 Processor -- start 2024-04-20T21:33:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 -- start start 2024-04-20T21:33:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 0x7f34081a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 0x7f34081a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 0x7f340819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3408118790 con 0x7f3408101640 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3408118610 con 0x7f3408106120 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3407200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3408118910 con 0x7f3408100d20 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 0x7f34081a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3406800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 0x7f340819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f3405e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 0x7f34081a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f33ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 0x7f34081a4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57772/0 (socket says 172.21.15.7:57772) 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.412+0000 7f33ffe00640 1 -- 172.21.15.7:0/3525501326 learned_addr learned my addr 172.21.15.7:0/3525501326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 -- 172.21.15.7:0/3525501326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 msgr2=0x7f34081a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 0x7f34081a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 -- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 msgr2=0x7f34081a4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 0x7f34081a4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 -- 172.21.15.7:0/3525501326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340819f1d0 con 0x7f3408106120 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.413+0000 7f3406800640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 0x7f340819ea50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f340002c510 tx=0x7f3400004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.414+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400043020 con 0x7f3408106120 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.414+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34000045f0 con 0x7f3408106120 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.414+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340819f460 con 0x7f3408106120 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.414+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340003e6b0 con 0x7f3408106120 2024-04-20T21:33:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.414+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34081ab760 con 0x7f3408106120 2024-04-20T21:33:32.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.415+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3408068800 con 0x7f3408106120 2024-04-20T21:33:32.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.417+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138872+0+0 (secure 0 0 0) 0x7f3400046020 con 0x7f3408106120 2024-04-20T21:33:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.418+0000 7f33fea00640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 0x7f33ec0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:33:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.418+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f34000f2ed0 con 0x7f3408106120 2024-04-20T21:33:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.418+0000 7f3405e00640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 0x7f33ec0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:33:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.419+0000 7f3405e00640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 0x7f33ec0a62e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f340819fb90 tx=0x7f33f8005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:33:32.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.422+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f340004d020 con 0x7f3408106120 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:32 smithi007 ceph-mon[16867]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-20T21:33:32.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.653+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f34081051f0 con 0x7f33ec0a3e20 2024-04-20T21:33:32.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.665+0000 7f33fea00640 1 -- 172.21.15.7:0/3525501326 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f34081051f0 con 0x7f33ec0a3e20 2024-04-20T21:33:32.664 INFO:teuthology.orchestra.run.smithi007.stdout:/ 1m 6m3h 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 msgr2=0x7f33ec0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 0x7f33ec0a62e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f340819fb90 tx=0x7f33f8005f90 comp rx=0 tx=0).stop 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 msgr2=0x7f340819ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 0x7f340819ea50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f340002c510 tx=0x7f3400004270 comp rx=0 tx=0).stop 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 shutdown_connections 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f33ec0a3e20 0x7f33ec0a62e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3408106120 0x7f340819ea50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408101640 0x7f34081a4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 --2- 172.21.15.7:0/3525501326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3408100d20 0x7f34081a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 >> 172.21.15.7:0/3525501326 conn(0x7f34080fc770 msgr2=0x7f3408103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 shutdown_connections 2024-04-20T21:33:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:33:32.668+0000 7f3407200640 1 -- 172.21.15.7:0/3525501326 wait complete. 2024-04-20T21:33:32.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-20T21:33:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:33:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:33:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:32 smithi016 ceph-mon[21409]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: from='client.25393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:32 smithi186 ceph-mon[22157]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-20T21:33:32.778 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-20T21:33:32.789 INFO:teuthology.run_tasks:Running task workunit... 2024-04-20T21:33:32.826 INFO:tasks.workunit:Pulling workunits from ref a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:33:32.826 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-20T21:33:32.826 INFO:tasks.workunit:timeout=3h 2024-04-20T21:33:32.826 INFO:tasks.workunit:cleanup=True 2024-04-20T21:33:32.827 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-20 21:31:01.489772637 +0000 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-20 21:31:01.489772637 +0000 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-20 21:33:19.372726892 +0000 2024-04-20T21:33:32.889 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-20 21:31:01.489772637 +0000 2024-04-20T21:33:32.889 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-20T21:33:32.890 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-20T21:33:32.995 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:33:33.055 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-20T21:33:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:33 smithi016 ceph-mon[21409]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:33 smithi016 ceph-mon[21409]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:33 smithi186 ceph-mon[22157]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:33 smithi186 ceph-mon[22157]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:33.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:33 smithi007 ceph-mon[16867]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:33.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:33 smithi007 ceph-mon[16867]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:34 smithi016 ceph-mon[21409]: from='client.25451 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:34 smithi016 ceph-mon[21409]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-20T21:33:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:34 smithi186 ceph-mon[22157]: from='client.25451 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:34 smithi186 ceph-mon[22157]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-20T21:33:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:34 smithi007 ceph-mon[16867]: from='client.25451 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:33:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:34 smithi007 ceph-mon[16867]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-04-20T21:33:35.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:35 smithi016 ceph-mon[21409]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:35 smithi186 ceph-mon[22157]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:35.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:35 smithi007 ceph-mon[16867]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:38.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:38 smithi016 ceph-mon[21409]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:38.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:38 smithi186 ceph-mon[22157]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:38.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:38 smithi007 ceph-mon[16867]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-20T21:33:40.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:40 smithi016 ceph-mon[21409]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-20T21:33:40.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:40 smithi186 ceph-mon[22157]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-20T21:33:40.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:40 smithi007 ceph-mon[16867]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-20T21:33:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:42 smithi016 ceph-mon[21409]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-20T21:33:42.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:42 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:33:42.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:42 smithi186 ceph-mon[22157]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-20T21:33:42.467 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:42 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:33:42.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:42 smithi007 ceph-mon[16867]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-20T21:33:42.536 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:42 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:33:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:44 smithi016 ceph-mon[21409]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-20T21:33:44.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:44 smithi186 ceph-mon[22157]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-20T21:33:44.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:44 smithi007 ceph-mon[16867]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-20T21:33:46.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:46 smithi186 ceph-mon[22157]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:46.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:46 smithi016 ceph-mon[21409]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:46.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:46 smithi007 ceph-mon[16867]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:48.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:48 smithi016 ceph-mon[21409]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:48.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:48 smithi186 ceph-mon[22157]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:48.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:48 smithi007 ceph-mon[16867]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-20T21:33:50.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:50 smithi016 ceph-mon[21409]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-20T21:33:50.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:50 smithi186 ceph-mon[22157]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-20T21:33:50.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:50 smithi007 ceph-mon[16867]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-20T21:33:52.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:52 smithi016 ceph-mon[21409]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:52.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:52 smithi186 ceph-mon[22157]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:52.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:52 smithi007 ceph-mon[16867]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:54.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:54 smithi016 ceph-mon[21409]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:54.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:54 smithi186 ceph-mon[22157]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:54.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:54 smithi007 ceph-mon[16867]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:33:56.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:56 smithi016 ceph-mon[21409]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:56.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:56 smithi186 ceph-mon[22157]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:56.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:56 smithi007 ceph-mon[16867]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:58.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:33:58 smithi016 ceph-mon[21409]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:58.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:33:58 smithi186 ceph-mon[22157]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:33:58.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:33:58 smithi007 ceph-mon[16867]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:00.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:00 smithi016 ceph-mon[21409]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:00.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:00 smithi186 ceph-mon[22157]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:00.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-mon[16867]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:01.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:01.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:34:00.708+0000 7f16b1000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:34:00.708+0000 7f16b1000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:34:00.708+0000 7f16b1000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:34:00.708+0000 7f16b1000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:01.033 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:34:00 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:34:00.708+0000 7f16b1000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-20T21:34:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 -- 172.21.15.7:0/897327038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13241041c0 msgr2=0x7f13241045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 --2- 172.21.15.7:0/897327038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13241041c0 0x7f13241045a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1314009900 tx=0x7f131402bc40 comp rx=0 tx=0).stop 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 -- 172.21.15.7:0/897327038 shutdown_connections 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 --2- 172.21.15.7:0/897327038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 0x7f1324110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 --2- 172.21.15.7:0/897327038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1324104b70 0x7f1324108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 --2- 172.21.15.7:0/897327038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f13241041c0 0x7f13241045a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.242+0000 7f1329600640 1 -- 172.21.15.7:0/897327038 >> 172.21.15.7:0/897327038 conn(0x7f13240ffa00 msgr2=0x7f1324101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.243+0000 7f1329600640 1 -- 172.21.15.7:0/897327038 shutdown_connections 2024-04-20T21:34:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.243+0000 7f1329600640 1 -- 172.21.15.7:0/897327038 wait complete. 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 Processor -- start 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 -- start start 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 0x7f13241a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f132419a770 0x7f132419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1324114370 con 0x7f13241041c0 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13241141f0 con 0x7f1324109290 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1329600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f13241144f0 con 0x7f132419a770 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1323e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1323e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47018/0 (socket says 172.21.15.7:47018) 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.244+0000 7f1323e00640 1 -- 172.21.15.7:0/2471955930 learned_addr learned my addr 172.21.15.7:0/2471955930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1328c00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f132419a770 0x7f132419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f131be00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 0x7f13241a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 -- 172.21.15.7:0/2471955930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f132419a770 msgr2=0x7f132419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f132419a770 0x7f132419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 -- 172.21.15.7:0/2471955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 msgr2=0x7f13241a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 0x7f13241a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 -- 172.21.15.7:0/2471955930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132419b370 con 0x7f13241041c0 2024-04-20T21:34:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.245+0000 7f1323e00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1314005ab0 tx=0x7f13140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.246+0000 7f1321600640 1 -- 172.21.15.7:0/2471955930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314041070 con 0x7f13241041c0 2024-04-20T21:34:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.246+0000 7f1321600640 1 -- 172.21.15.7:0/2471955930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131402cca0 con 0x7f13241041c0 2024-04-20T21:34:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.246+0000 7f1321600640 1 -- 172.21.15.7:0/2471955930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314003d90 con 0x7f13241041c0 2024-04-20T21:34:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.246+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132419b600 con 0x7f13241041c0 2024-04-20T21:34:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.246+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13241a7740 con 0x7f13241041c0 2024-04-20T21:34:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.249+0000 7f1321600640 1 -- 172.21.15.7:0/2471955930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f1314044070 con 0x7f13241041c0 2024-04-20T21:34:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.249+0000 7f132da11740 1 Processor -- start 2024-04-20T21:34:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 -- start start 2024-04-20T21:34:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x563100307760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307ca0 0x563100310340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x563100312c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56310026d150 con 0x563100310880 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563100213370 con 0x563100307ca0 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f132da11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563100269110 con 0x563100307380 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f131be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x563100312c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f131be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x563100312c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47020/0 (socket says 172.21.15.7:47020) 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f131be00640 1 -- 172.21.15.7:0/1479551151 learned_addr learned my addr 172.21.15.7:0/1479551151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f1323e00640 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307ca0 0x563100310340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.250+0000 7f1328c00640 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x563100307760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 -- 172.21.15.7:0/1479551151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307ca0 msgr2=0x563100310340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307ca0 0x563100310340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 -- 172.21.15.7:0/1479551151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 msgr2=0x563100312c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x563100312c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 -- 172.21.15.7:0/1479551151 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631000497d0 con 0x563100307380 2024-04-20T21:34:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1328c00640 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x563100307760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f131c009f00 tx=0x7f131c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5d97c935bac10b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1321600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 0x7f13040a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1319200640 1 -- 172.21.15.7:0/1479551151 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c004d20 con 0x563100307380 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1319200640 1 -- 172.21.15.7:0/1479551151 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c004050 con 0x563100307380 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.251+0000 7f1319200640 1 -- 172.21.15.7:0/1479551151 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c005280 con 0x563100307380 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f1321600640 1 -- 172.21.15.7:0/2471955930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f13140f2220 con 0x7f13241041c0 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f1323e00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 0x7f13040a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f1323e00640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 0x7f13040a6410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13140ef8f0 tx=0x7f13140f5cd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 -- 172.21.15.7:0/1479551151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 msgr2=0x563100307760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x563100307760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f131c009f00 tx=0x7f131c011810 comp rx=0 tx=0).stop 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 -- 172.21.15.7:0/1479551151 shutdown_connections 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307ca0 0x563100310340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x563100312c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 --2- 172.21.15.7:0/1479551151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x563100307760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 -- 172.21.15.7:0/1479551151 >> 172.21.15.7:0/1479551151 conn(0x563100301ab0 msgr2=0x563100301e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 -- 172.21.15.7:0/1479551151 shutdown_connections 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.252+0000 7f132da11740 1 -- 172.21.15.7:0/1479551151 wait complete. 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.253+0000 7f132da11740 1 -- start start 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.253+0000 7f132da11740 1 Processor -- start 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x56310030d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x56310030e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563100313670 con 0x563100307ca0 2024-04-20T21:34:01.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563100226740 con 0x563100310880 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5631003137f0 con 0x563100307380 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f132da11740 10 client.0 _ll_register_callbacks cb 0x563100221b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47034/0 (socket says 172.21.15.7:47034) 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 -- 172.21.15.7:0/4014772028 learned_addr learned my addr 172.21.15.7:0/4014772028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f1328c00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x56310030d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f1328c00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x56310030e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 -- 172.21.15.7:0/4014772028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 msgr2=0x56310030d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x56310030d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 -- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 msgr2=0x56310030e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x56310030e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.254+0000 7f131be00640 1 -- 172.21.15.7:0/4014772028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630fff7f3f0 con 0x563100307ca0 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f1328c00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x56310030e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f131be00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f130c012710 tx=0x7f130c012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd85b2daf32f3805 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f1311600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c004020 con 0x563100307ca0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f130c0041c0 con 0x563100307ca0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c004050 con 0x563100307ca0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f130c0043c0 con 0x563100307ca0 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f132da11740 10 client.15354 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:01.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.255+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5630fff3f0f0 con 0x563100307ca0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c004050 con 0x563100307ca0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f130c003af0 con 0x563100307ca0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 10 client.15354 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 20 client.15354 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100320c00 0x563100323040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 4 client.15354 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.256+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56310027ab70 con 0x563100320c00 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.257+0000 7f1328c00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100320c00 0x563100323040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.257+0000 7f1328c00640 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100320c00 0x563100323040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f457d749987d52 server_cookie=6f6af0e9ee331c2c in_seq=0 out_seq=0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.257+0000 7f1311600640 10 client.15354 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f1311600640 1 -- 172.21.15.7:0/4014772028 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56310027ab70 con 0x563100320c00 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f1311600640 10 client.15354 handle_command_reply: tid=0 2024-04-20T21:34:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 client.15354 shutdown 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 msgr2=0x56310030ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f130c012710 tx=0x7f130c012bd0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 shutdown_connections 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x56310030e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100320c00 0x563100323040 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307ca0 0x56310030ddc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 --2- 172.21.15.7:0/4014772028 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307380 0x56310030d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f1311600640 0 client.15354 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.258+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 >> 172.21.15.7:0/4014772028 conn(0x563100301ab0 msgr2=0x563100303800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.259+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 shutdown_connections 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.259+0000 7f132da11740 1 -- 172.21.15.7:0/4014772028 wait complete. 2024-04-20T21:34:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.259+0000 7f132da11740 20 client.15354 trim_cache size 0 max 16384 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 Processor -- start 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 -- start start 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 0x56310031c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x56310030f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13241b24e0 con 0x563100310880 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13241afda0 con 0x563100307380 2024-04-20T21:34:01.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.278+0000 7f132da11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f13241adc10 con 0x563100307ca0 2024-04-20T21:34:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f1328c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 0x56310031c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f1328c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x56310030f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:51138/0 (socket says 172.21.15.7:51138) 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 -- 172.21.15.7:0/1528373535 learned_addr learned my addr 172.21.15.7:0/1528373535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 -- 172.21.15.7:0/1528373535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 msgr2=0x56310031c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 0x56310031c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 -- 172.21.15.7:0/1528373535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 msgr2=0x56310030f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x56310030f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f131be00640 1 -- 172.21.15.7:0/1528373535 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563100312c70 con 0x563100307ca0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.279+0000 7f1328c00640 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 0x56310031c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.280+0000 7f131be00640 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f130c00f810 tx=0x7f130c009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe90746501552b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.280+0000 7f1313e00640 1 -- 172.21.15.7:0/1528373535 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c004020 con 0x563100307ca0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.280+0000 7f1313e00640 1 -- 172.21.15.7:0/1528373535 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c0041c0 con 0x563100307ca0 2024-04-20T21:34:01.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.280+0000 7f1313e00640 1 -- 172.21.15.7:0/1528373535 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c00fe30 con 0x563100307ca0 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- 172.21.15.7:0/1528373535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 msgr2=0x56310030d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f130c00f810 tx=0x7f130c009850 comp rx=0 tx=0).stop 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- 172.21.15.7:0/1528373535 shutdown_connections 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100310880 0x56310030f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x56310030d160 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 --2- 172.21.15.7:0/1528373535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100307380 0x56310031c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- 172.21.15.7:0/1528373535 >> 172.21.15.7:0/1528373535 conn(0x563100301ab0 msgr2=0x56310030efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- 172.21.15.7:0/1528373535 shutdown_connections 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- 172.21.15.7:0/1528373535 wait complete. 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.281+0000 7f132da11740 1 -- start start 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 Processor -- start 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x563100314110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x563100314650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13241ae260 con 0x563100307380 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13241af6d0 con 0x563100310880 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f13241ae930 con 0x563100307ca0 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f132da11740 10 client.0 _ll_register_callbacks cb 0x7f130c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.282+0000 7f1328c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47060/0 (socket says 172.21.15.7:47060) 2024-04-20T21:34:01.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 -- 172.21.15.7:0/3848255631 learned_addr learned my addr 172.21.15.7:0/3848255631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x563100314650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f131be00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x563100314110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 -- 172.21.15.7:0/3848255631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 msgr2=0x563100314110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x563100314110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 -- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 msgr2=0x563100314650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x563100314650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 -- 172.21.15.7:0/3848255631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563100314e50 con 0x563100307380 2024-04-20T21:34:01.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f131be00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x563100314110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1328c00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f131c009ed0 tx=0x7f131c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dd68fafcbbb54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1311600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.283+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c02ee60 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130c0041c0 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f131c003c40 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f131c03f050 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f132da11740 10 client.15360 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563100387150 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130c0041c0 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.284+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f131c02c160 con 0x563100307380 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 10 client.15360 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:01.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 20 client.15360 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100387690 0x563100389a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 4 client.15360 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631002749e0 con 0x563100387690 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f131be00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100387690 0x563100389a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f131be00640 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100387690 0x563100389a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2955da96e1e4867 server_cookie=88c4b47f313b818b in_seq=0 out_seq=0 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.285+0000 7f1311600640 10 client.15360 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f1311600640 1 -- 172.21.15.7:0/3848255631 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631002749e0 con 0x563100387690 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f1311600640 10 client.15360 handle_command_reply: tid=0 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 client.15360 shutdown 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 msgr2=0x563100318da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f131c009ed0 tx=0x7f131c009500 comp rx=0 tx=0).stop 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 shutdown_connections 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563100310880 0x563100314650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563100387690 0x563100389a80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563100307ca0 0x563100314110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 --2- 172.21.15.7:0/3848255631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563100307380 0x563100318da0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 >> 172.21.15.7:0/3848255631 conn(0x563100301ab0 msgr2=0x56310030f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f1311600640 0 client.15360 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 shutdown_connections 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 1 -- 172.21.15.7:0/3848255631 wait complete. 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.286+0000 7f132da11740 20 client.15360 trim_cache size 0 max 16384 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:34:01.288 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 msgr2=0x7f13040a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 0x7f13040a6410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13140ef8f0 tx=0x7f13140f5cd0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 msgr2=0x7f13241a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1314005ab0 tx=0x7f13140042f0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 shutdown_connections 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f13040a3f50 0x7f13040a6410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1324109290 0x7f13241a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f132419a770 0x7f132419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 --2- 172.21.15.7:0/2471955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13241041c0 0x7f13241a0660 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 >> 172.21.15.7:0/2471955930 conn(0x7f13240ffa00 msgr2=0x7f1324100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 shutdown_connections 2024-04-20T21:34:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.287+0000 7f1329600640 1 -- 172.21.15.7:0/2471955930 wait complete. 2024-04-20T21:34:01.303 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:34:01.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 -- 172.21.15.7:0/2744773997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874109370 msgr2=0x7fe874110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 --2- 172.21.15.7:0/2744773997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874109370 0x7fe874110f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009ca0 tx=0x7fe86c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 -- 172.21.15.7:0/2744773997 shutdown_connections 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 --2- 172.21.15.7:0/2744773997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe874109370 0x7fe874110f40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 --2- 172.21.15.7:0/2744773997 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe874104c50 0x7fe874108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 --2- 172.21.15.7:0/2744773997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8741042a0 0x7fe874104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 -- 172.21.15.7:0/2744773997 >> 172.21.15.7:0/2744773997 conn(0x7fe8740ffae0 msgr2=0x7fe874101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 -- 172.21.15.7:0/2744773997 shutdown_connections 2024-04-20T21:34:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.563+0000 7fe87a200640 1 -- 172.21.15.7:0/2744773997 wait complete. 2024-04-20T21:34:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 Processor -- start 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 -- start start 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 0x7fe8741a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe874109370 0x7fe8741a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 0x7fe87419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114410 con 0x7fe874109370 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114290 con 0x7fe87419a750 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.564+0000 7fe87a200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe874114590 con 0x7fe8741042a0 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe879800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 0x7fe87419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe879800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 0x7fe87419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45708/0 (socket says 172.21.15.7:45708) 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe879800640 1 -- 172.21.15.7:0/190950378 learned_addr learned my addr 172.21.15.7:0/190950378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 0x7fe8741a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe873e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe874109370 0x7fe8741a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 -- 172.21.15.7:0/190950378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 msgr2=0x7fe87419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 0x7fe87419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 -- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe874109370 msgr2=0x7fe8741a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe874109370 0x7fe8741a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.565+0000 7fe878e00640 1 -- 172.21.15.7:0/190950378 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87419b350 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe878e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 0x7fe8741a0690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800cc40 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741a7430 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86800cde0 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868018920 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.566+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741a7890 con 0x7fe8741042a0 2024-04-20T21:34:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.568+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874105770 con 0x7fe8741042a0 2024-04-20T21:34:01.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.569+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fe868018ac0 con 0x7fe8741042a0 2024-04-20T21:34:01.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.571+0000 7fe871600640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 0x7fe85c0a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:01.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.571+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe8680d0d40 con 0x7fe8741042a0 2024-04-20T21:34:01.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.571+0000 7fe873e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 0x7fe85c0a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:01.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.572+0000 7fe873e00640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 0x7fe85c0a63d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8640095f0 tx=0x7fe864005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:01.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.575+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe86809cba0 con 0x7fe8741042a0 2024-04-20T21:34:01.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.825+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe87410c600 con 0x7fe8741042a0 2024-04-20T21:34:01.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.827+0000 7fe871600640 1 -- 172.21.15.7:0/190950378 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7fe8680a0ea0 con 0x7fe8741042a0 2024-04-20T21:34:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:34:01.826 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:33:20.441301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:34:01.826 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-04-20T21:34:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.829+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 msgr2=0x7fe85c0a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.829+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 0x7fe85c0a63d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8640095f0 tx=0x7fe864005f90 comp rx=0 tx=0).stop 2024-04-20T21:34:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.829+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 msgr2=0x7fe8741a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.829+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 0x7fe8741a0690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe86800e9f0 tx=0x7fe86800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 shutdown_connections 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe85c0a3f10 0x7fe85c0a63d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe87419a750 0x7fe87419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe874109370 0x7fe8741a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 --2- 172.21.15.7:0/190950378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8741042a0 0x7fe8741a0690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 >> 172.21.15.7:0/190950378 conn(0x7fe8740ffae0 msgr2=0x7fe874101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 shutdown_connections 2024-04-20T21:34:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:01.830+0000 7fe87a200640 1 -- 172.21.15.7:0/190950378 wait complete. 2024-04-20T21:34:01.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 -- 172.21.15.7:0/2330202174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 msgr2=0x7f9c081045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 --2- 172.21.15.7:0/2330202174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081045d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009900 tx=0x7f9bf802bc40 comp rx=0 tx=0).stop 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 -- 172.21.15.7:0/2330202174 shutdown_connections 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 --2- 172.21.15.7:0/2330202174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 0x7f9c08110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 --2- 172.21.15.7:0/2330202174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c08104ba0 0x7f9c08108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 --2- 172.21.15.7:0/2330202174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081045d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 -- 172.21.15.7:0/2330202174 >> 172.21.15.7:0/2330202174 conn(0x7f9c080ffa50 msgr2=0x7f9c08101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.066+0000 7f9c0d000640 1 -- 172.21.15.7:0/2330202174 shutdown_connections 2024-04-20T21:34:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.067+0000 7f9c0d000640 1 -- 172.21.15.7:0/2330202174 wait complete. 2024-04-20T21:34:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.067+0000 7f9c0d000640 1 Processor -- start 2024-04-20T21:34:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 -- start start 2024-04-20T21:34:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 0x7f9c081a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c0819a700 0x7f9c0819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081143a0 con 0x7f9c0819a700 2024-04-20T21:34:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08114220 con 0x7f9c081041f0 2024-04-20T21:34:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c0d000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c08114520 con 0x7f9c081092c0 2024-04-20T21:34:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c07400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c07400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45722/0 (socket says 172.21.15.7:45722) 2024-04-20T21:34:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c07400640 1 -- 172.21.15.7:0/425246684 learned_addr learned my addr 172.21.15.7:0/425246684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c07e00640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c0819a700 0x7f9c0819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.068+0000 7f9c06a00640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 0x7f9c081a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 -- 172.21.15.7:0/425246684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 msgr2=0x7f9c081a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 0x7f9c081a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 -- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c0819a700 msgr2=0x7f9c0819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c0819a700 0x7f9c0819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 -- 172.21.15.7:0/425246684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0819b300 con 0x7f9c081041f0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.069+0000 7f9c07400640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bf802efd0 tx=0x7f9bf80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.070+0000 7f9bffe00640 1 -- 172.21.15.7:0/425246684 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8041070 con 0x7f9c081041f0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.070+0000 7f9bffe00640 1 -- 172.21.15.7:0/425246684 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf80054e0 con 0x7f9c081041f0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.070+0000 7f9bffe00640 1 -- 172.21.15.7:0/425246684 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8034440 con 0x7f9c081041f0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.070+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0819b590 con 0x7f9c081041f0 2024-04-20T21:34:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.070+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c081a7720 con 0x7f9c081041f0 2024-04-20T21:34:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9c1136e740 1 Processor -- start 2024-04-20T21:34:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9c1136e740 1 -- start start 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9bffe00640 1 -- 172.21.15.7:0/425246684 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f9bf8044070 con 0x7f9c081041f0 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.073+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c1136e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5633cd637050 con 0x5633cd6d2170 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c1136e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633cd5dd490 con 0x5633cd6d2a90 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c1136e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5633cd633010 con 0x5633cd6db670 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47092/0 (socket says 172.21.15.7:47092) 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 -- 172.21.15.7:0/3677362059 learned_addr learned my addr 172.21.15.7:0/3677362059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07400640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c06a00640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 -- 172.21.15.7:0/3677362059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 msgr2=0x5633cd6dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 -- 172.21.15.7:0/3677362059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 msgr2=0x5633cd6db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6db130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c07e00640 1 -- 172.21.15.7:0/3677362059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633cd5e5070 con 0x5633cd6d2170 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.074+0000 7f9c06a00640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:02.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.075+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009f70 tx=0x7f9c00010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64b6fcaac05a7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.075+0000 7f9bfcc00640 1 -- 172.21.15.7:0/3677362059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00016070 con 0x5633cd6d2170 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.075+0000 7f9bfcc00640 1 -- 172.21.15.7:0/3677362059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c00004d20 con 0x5633cd6d2170 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.075+0000 7f9bfcc00640 1 -- 172.21.15.7:0/3677362059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00012930 con 0x5633cd6d2170 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9bffe00640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 0x7f9bec0a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 -- 172.21.15.7:0/3677362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 msgr2=0x5633cd6d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009f70 tx=0x7f9c00010800 comp rx=0 tx=0).stop 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 -- 172.21.15.7:0/3677362059 shutdown_connections 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3677362059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d2550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 -- 172.21.15.7:0/3677362059 >> 172.21.15.7:0/3677362059 conn(0x5633cd6cc8a0 msgr2=0x5633cd6ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c07400640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 0x7f9bec0a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9bffe00640 1 -- 172.21.15.7:0/425246684 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9bf80f2410 con 0x7f9c081041f0 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 -- 172.21.15.7:0/3677362059 shutdown_connections 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.076+0000 7f9c1136e740 1 -- 172.21.15.7:0/3677362059 wait complete. 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.077+0000 7f9c07400640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 0x7f9bec0a6440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80efae0 tx=0x7f9bf80f5ec0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.077+0000 7f9c1136e740 1 -- start start 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.077+0000 7f9c1136e740 1 Processor -- start 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.077+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.077+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c1136e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5633cd6de460 con 0x5633cd6d2170 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c1136e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5633cd5f0860 con 0x5633cd6d2a90 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c1136e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5633cd6de5e0 con 0x5633cd6db670 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c1136e740 10 client.0 _ll_register_callbacks cb 0x5633cd5befe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47094/0 (socket says 172.21.15.7:47094) 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 -- 172.21.15.7:0/314064936 learned_addr learned my addr 172.21.15.7:0/314064936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c06a00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 -- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 msgr2=0x5633cd6d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 -- 172.21.15.7:0/314064936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 msgr2=0x5633cd6d8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6d8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.078+0000 7f9c07e00640 1 -- 172.21.15.7:0/314064936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633cd5f4950 con 0x5633cd6d2170 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9c07e00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011010 tx=0x7f9c00012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba9134a23c02811 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00016070 con 0x5633cd6d2170 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9c00022040 con 0x5633cd6d2170 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c00004d20 con 0x5633cd6d2170 2024-04-20T21:34:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9c000050e0 con 0x5633cd6d2170 2024-04-20T21:34:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9c1136e740 10 client.15378 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5633cd5c2820 con 0x5633cd6d2170 2024-04-20T21:34:02.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.079+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c00004d20 con 0x5633cd6d2170 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f9c0000f040 con 0x5633cd6d2170 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 10 client.15378 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 10 client.15378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 20 client.15378 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd6eb550 0x5633cd6ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 4 client.15378 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633cd644a70 con 0x5633cd6eb550 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.080+0000 7f9c06a00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd6eb550 0x5633cd6ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.081+0000 7f9c06a00640 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd6eb550 0x5633cd6ed940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92a200e8494d9ea9 server_cookie=cbebad1b0bb83c77 in_seq=0 out_seq=0 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.081+0000 7f9bf1600640 10 client.15378 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9bf1600640 1 -- 172.21.15.7:0/314064936 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633cd644a70 con 0x5633cd6eb550 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9bf1600640 10 client.15378 handle_command_reply: tid=0 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 client.15378 shutdown 2024-04-20T21:34:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 msgr2=0x5633cd6d8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011010 tx=0x7f9c00012590 comp rx=0 tx=0).stop 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 shutdown_connections 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd6eb550 0x5633cd6ed940 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2a90 0x5633cd6d8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 --2- 172.21.15.7:0/314064936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2170 0x5633cd6d8700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 >> 172.21.15.7:0/314064936 conn(0x5633cd6cc8a0 msgr2=0x5633cd6ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9bf1600640 0 client.15378 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 shutdown_connections 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 1 -- 172.21.15.7:0/314064936 wait complete. 2024-04-20T21:34:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.082+0000 7f9c1136e740 20 client.15378 trim_cache size 0 max 16384 2024-04-20T21:34:02.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 Processor -- start 2024-04-20T21:34:02.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 -- start start 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6d2a90 0x5633cd6d7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6db670 0x5633cd6e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081b2550 con 0x5633cd6db670 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081afe10 con 0x5633cd6d2170 2024-04-20T21:34:02.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c1136e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081adc80 con 0x5633cd6d2a90 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45752/0 (socket says 172.21.15.7:45752) 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c07e00640 1 -- 172.21.15.7:0/1766010360 learned_addr learned my addr 172.21.15.7:0/1766010360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.098+0000 7f9c07e00640 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6db670 0x5633cd6e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c06a00640 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6d2a90 0x5633cd6d7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 -- 172.21.15.7:0/1766010360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6d2a90 msgr2=0x5633cd6d7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6d2a90 0x5633cd6d7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 -- 172.21.15.7:0/1766010360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6db670 msgr2=0x5633cd6e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6db670 0x5633cd6e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 -- 172.21.15.7:0/1766010360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633cd6d28c0 con 0x5633cd6d2170 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9c07e00640 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c00002230 tx=0x7f9c00010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe3f8ec2047beb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9bf3e00640 1 -- 172.21.15.7:0/1766010360 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00016070 con 0x5633cd6d2170 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9bf3e00640 1 -- 172.21.15.7:0/1766010360 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c00010d00 con 0x5633cd6d2170 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.099+0000 7f9bf3e00640 1 -- 172.21.15.7:0/1766010360 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00005b60 con 0x5633cd6d2170 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 -- 172.21.15.7:0/1766010360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 msgr2=0x5633cd6dae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9c00002230 tx=0x7f9c00010800 comp rx=0 tx=0).stop 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 -- 172.21.15.7:0/1766010360 shutdown_connections 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6db670 0x5633cd6e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6d2a90 0x5633cd6d7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 --2- 172.21.15.7:0/1766010360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6dae30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 -- 172.21.15.7:0/1766010360 >> 172.21.15.7:0/1766010360 conn(0x5633cd6cc8a0 msgr2=0x5633cd6cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 -- 172.21.15.7:0/1766010360 shutdown_connections 2024-04-20T21:34:02.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.100+0000 7f9c1136e740 1 -- 172.21.15.7:0/1766010360 wait complete. 2024-04-20T21:34:02.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 -- start start 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 Processor -- start 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ae2d0 con 0x5633cd6d2a90 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081af740 con 0x5633cd6d2170 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9c081ae9a0 con 0x5633cd6db670 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c1136e740 10 client.0 _ll_register_callbacks cb 0x7f9c00004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.101+0000 7f9c06a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47102/0 (socket says 172.21.15.7:47102) 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 -- 172.21.15.7:0/3342249017 learned_addr learned my addr 172.21.15.7:0/3342249017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 -- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 msgr2=0x5633cd6df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 -- 172.21.15.7:0/3342249017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 msgr2=0x5633cd6e3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6e3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 -- 172.21.15.7:0/3342249017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633cd3493f0 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9c06a00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005d00 tx=0x7f9bf4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fcfac56d775f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.102+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4004220 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9bf40043c0 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c00010d00 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9bf402c050 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9c1136e740 10 client.15384 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5633cd4137d0 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c00010d00 con 0x5633cd6d2a90 2024-04-20T21:34:02.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.103+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f9bf4011440 con 0x5633cd6d2a90 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 10 client.15384 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 10 client.15384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 20 client.15384 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd751e50 0x5633cd754270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 4 client.15384 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633cd63e8e0 con 0x5633cd751e50 2024-04-20T21:34:02.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd751e50 0x5633cd754270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9c07e00640 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd751e50 0x5633cd754270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1134ce497ed3910c server_cookie=cf0ca0424fabbf98 in_seq=0 out_seq=0 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.104+0000 7f9bf1600640 10 client.15384 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9bf1600640 1 -- 172.21.15.7:0/3342249017 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633cd63e8e0 con 0x5633cd751e50 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9bf1600640 10 client.15384 handle_command_reply: tid=0 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 client.15384 shutdown 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 msgr2=0x5633cd6df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4005d00 tx=0x7f9bf4004e20 comp rx=0 tx=0).stop 2024-04-20T21:34:02.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 shutdown_connections 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5633cd6db670 0x5633cd6df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5633cd6d2a90 0x5633cd6df130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5633cd751e50 0x5633cd754270 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 --2- 172.21.15.7:0/3342249017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5633cd6d2170 0x5633cd6e3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 >> 172.21.15.7:0/3342249017 conn(0x5633cd6cc8a0 msgr2=0x5633cd6cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9bf1600640 0 client.15384 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 shutdown_connections 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 1 -- 172.21.15.7:0/3342249017 wait complete. 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.105+0000 7f9c1136e740 20 client.15384 trim_cache size 0 max 16384 2024-04-20T21:34:02.107 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 msgr2=0x7f9bec0a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 0x7f9bec0a6440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80efae0 tx=0x7f9bf80f5ec0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 msgr2=0x7f9c081a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bf802efd0 tx=0x7f9bf80045e0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 shutdown_connections 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c0819a700 0x7f9c0819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c081092c0 0x7f9c081a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9bec0a3f80 0x7f9bec0a6440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 --2- 172.21.15.7:0/425246684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c081041f0 0x7f9c081a0640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 >> 172.21.15.7:0/425246684 conn(0x7f9c080ffa50 msgr2=0x7f9c08100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 shutdown_connections 2024-04-20T21:34:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.106+0000 7f9c0d000640 1 -- 172.21.15.7:0/425246684 wait complete. 2024-04-20T21:34:02.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:34:02.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 -- 172.21.15.7:0/649769385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48105d70 msgr2=0x7fbc481061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/649769385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48105d70 0x7fbc481061d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009890 tx=0x7fbc3802bb70 comp rx=0 tx=0).stop 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 -- 172.21.15.7:0/649769385 shutdown_connections 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/649769385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 0x7fbc48111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/649769385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48105d70 0x7fbc481061d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/649769385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48104490 0x7fbc48104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.343+0000 7fbc4fa00640 1 -- 172.21.15.7:0/649769385 >> 172.21.15.7:0/649769385 conn(0x7fbc480ffbf0 msgr2=0x7fbc48102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.344+0000 7fbc4fa00640 1 -- 172.21.15.7:0/649769385 shutdown_connections 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.344+0000 7fbc4fa00640 1 -- 172.21.15.7:0/649769385 wait complete. 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.345+0000 7fbc4fa00640 1 Processor -- start 2024-04-20T21:34:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.345+0000 7fbc4fa00640 1 -- start start 2024-04-20T21:34:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.345+0000 7fbc4fa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 0x7fbc481a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4fa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 0x7fbc4819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4fa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48114600 con 0x7fbc48105d70 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4fa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48114480 con 0x7fbc48106710 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4fa00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc48114780 con 0x7fbc48104490 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 0x7fbc481a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47114/0 (socket says 172.21.15.7:47114) 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc47e00640 1 -- 172.21.15.7:0/4138027222 learned_addr learned my addr 172.21.15.7:0/4138027222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4da00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 0x7fbc4819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 -- 172.21.15.7:0/4138027222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 msgr2=0x7fbc4819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 0x7fbc4819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 -- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 msgr2=0x7fbc481a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc4d000640 1 -- 172.21.15.7:0/4138027222 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4819b0c0 con 0x7fbc48104490 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.346+0000 7fbc47e00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc4d000640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 0x7fbc481a0740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e790 tx=0x7fbc3c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc45600640 1 -- 172.21.15.7:0/4138027222 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00eea0 con 0x7fbc48104490 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc45600640 1 -- 172.21.15.7:0/4138027222 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c00ccc0 con 0x7fbc48104490 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc45600640 1 -- 172.21.15.7:0/4138027222 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c018780 con 0x7fbc48104490 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4819b3b0 con 0x7fbc48104490 2024-04-20T21:34:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.347+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a7600 con 0x7fbc48104490 2024-04-20T21:34:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 Processor -- start 2024-04-20T21:34:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 -- start start 2024-04-20T21:34:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc45600640 1 -- 172.21.15.7:0/4138027222 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fbc3c010430 con 0x7fbc48104490 2024-04-20T21:34:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 0x5582bb93ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 0x5582bb9428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5582bb89c050 con 0x5582bb9404b0 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb842490 con 0x5582bb9378d0 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.352+0000 7fbc527cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5582bb898010 con 0x5582bb936fb0 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 0x5582bb93ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:51252/0 (socket says 172.21.15.7:51252) 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 0x5582bb93ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45798/0 (socket says 172.21.15.7:45798) 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 -- 172.21.15.7:0/2827288444 learned_addr learned my addr 172.21.15.7:0/2827288444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc47e00640 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 0x5582bb9428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 -- 172.21.15.7:0/2827288444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 msgr2=0x5582bb93ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 0x5582bb93ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 -- 172.21.15.7:0/2827288444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 msgr2=0x5582bb9428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 0x5582bb9428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc4da00640 1 -- 172.21.15.7:0/2827288444 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bb84a070 con 0x5582bb936fb0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.353+0000 7fbc47e00640 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 0x5582bb9428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.354+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009fd0 tx=0x7fbc400118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4351ca3e51b9af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.354+0000 7fbc36000640 1 -- 172.21.15.7:0/2827288444 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40004d20 con 0x5582bb936fb0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.354+0000 7fbc36000640 1 -- 172.21.15.7:0/2827288444 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40004050 con 0x5582bb936fb0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.354+0000 7fbc36000640 1 -- 172.21.15.7:0/2827288444 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40005220 con 0x5582bb936fb0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.354+0000 7fbc45600640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 0x7fbc300a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc45600640 1 -- 172.21.15.7:0/4138027222 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbc3c0d0bc0 con 0x7fbc48104490 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc4d000640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 0x7fbc300a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 -- 172.21.15.7:0/2827288444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 msgr2=0x5582bb937390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009fd0 tx=0x7fbc400118d0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 -- 172.21.15.7:0/2827288444 shutdown_connections 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9378d0 0x5582bb93ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9404b0 0x5582bb9428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2827288444 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb937390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 -- 172.21.15.7:0/2827288444 >> 172.21.15.7:0/2827288444 conn(0x5582bb9316e0 msgr2=0x5582bb931ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 -- 172.21.15.7:0/2827288444 shutdown_connections 2024-04-20T21:34:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.355+0000 7fbc527cc740 1 -- 172.21.15.7:0/2827288444 wait complete. 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc4d000640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 0x7fbc300a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ce180 tx=0x7fbc3c0d4690 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 -- start start 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 Processor -- start 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb93d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9404b0 0x5582bb93df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5582bb9432a0 con 0x5582bb9378d0 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582bb855860 con 0x5582bb9404b0 2024-04-20T21:34:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5582bb943420 con 0x5582bb936fb0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.356+0000 7fbc527cc740 10 client.0 _ll_register_callbacks cb 0x5582bb823fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc4da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb93d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47118/0 (socket says 172.21.15.7:47118) 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 -- 172.21.15.7:0/2958538633 learned_addr learned my addr 172.21.15.7:0/2958538633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9404b0 0x5582bb93df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 -- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 msgr2=0x5582bb93d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb93d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 -- 172.21.15.7:0/2958538633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9404b0 msgr2=0x5582bb93df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9404b0 0x5582bb93df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.357+0000 7fbc47e00640 1 -- 172.21.15.7:0/2958538633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bb859950 con 0x5582bb9378d0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc47e00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc38037040 tx=0x7fbc38004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a0abab904e2c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38044070 con 0x5582bb9378d0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbc38034640 con 0x5582bb9378d0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc40004050 con 0x5582bb9378d0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbc38050050 con 0x5582bb9378d0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc527cc740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.358+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5582bb827820 con 0x5582bb9378d0 2024-04-20T21:34:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40004050 con 0x5582bb9378d0 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fbc380328a0 con 0x5582bb9378d0 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 10 client.15396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 20 client.15396 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb950470 0x5582bb952890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 4 client.15396 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582bb8a9a70 con 0x5582bb950470 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.359+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb950470 0x5582bb952890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.360+0000 7fbc4da00640 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb950470 0x5582bb952890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d34b5ebba546bdec server_cookie=b2736904a459b8 in_seq=0 out_seq=0 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.360+0000 7fbc2aa00640 10 client.15396 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc2aa00640 1 -- 172.21.15.7:0/2958538633 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582bb8a9a70 con 0x5582bb950470 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc2aa00640 10 client.15396 handle_command_reply: tid=0 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 client.15396 shutdown 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 msgr2=0x5582bb93da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc38037040 tx=0x7fbc38004560 comp rx=0 tx=0).stop 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 shutdown_connections 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb950470 0x5582bb952890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb9404b0 0x5582bb93df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93da50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 --2- 172.21.15.7:0/2958538633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb936fb0 0x5582bb93d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 >> 172.21.15.7:0/2958538633 conn(0x5582bb9316e0 msgr2=0x5582bb933430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc2aa00640 0 client.15396 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 shutdown_connections 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.361+0000 7fbc527cc740 1 -- 172.21.15.7:0/2958538633 wait complete. 2024-04-20T21:34:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.362+0000 7fbc527cc740 20 client.15396 trim_cache size 0 max 16384 2024-04-20T21:34:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.376+0000 7fbc527cc740 1 Processor -- start 2024-04-20T21:34:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.376+0000 7fbc527cc740 1 -- start start 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb958160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481b25c0 con 0x5582bb9378d0 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481afe80 con 0x5582bb936fb0 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc527cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481adcd0 con 0x5582bb9404b0 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc4da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc4da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45802/0 (socket says 172.21.15.7:45802) 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc4da00640 1 -- 172.21.15.7:0/3608499098 learned_addr learned my addr 172.21.15.7:0/3608499098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc4da00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb958160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.377+0000 7fbc47e00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 -- 172.21.15.7:0/3608499098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 msgr2=0x5582bb958160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb958160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 -- 172.21.15.7:0/3608499098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 msgr2=0x5582bb93ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 -- 172.21.15.7:0/3608499098 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bb5ae3f0 con 0x5582bb936fb0 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc47e00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc4da00640 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc40014040 tx=0x7fbc400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60b6a5a56bccbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc35600640 1 -- 172.21.15.7:0/3608499098 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40004050 con 0x5582bb936fb0 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc35600640 1 -- 172.21.15.7:0/3608499098 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc400046d0 con 0x5582bb936fb0 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.378+0000 7fbc35600640 1 -- 172.21.15.7:0/3608499098 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000fd60 con 0x5582bb936fb0 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 -- 172.21.15.7:0/3608499098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 msgr2=0x5582bb94c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc40014040 tx=0x7fbc400118d0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 -- 172.21.15.7:0/3608499098 shutdown_connections 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb958160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb93ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 --2- 172.21.15.7:0/3608499098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb94c160 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 -- 172.21.15.7:0/3608499098 >> 172.21.15.7:0/3608499098 conn(0x5582bb9316e0 msgr2=0x5582bb930740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 -- 172.21.15.7:0/3608499098 shutdown_connections 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.379+0000 7fbc527cc740 1 -- 172.21.15.7:0/3608499098 wait complete. 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 -- start start 2024-04-20T21:34:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 Processor -- start 2024-04-20T21:34:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb948ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb9443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ae340 con 0x5582bb9378d0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481af7b0 con 0x5582bb936fb0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481aea10 con 0x5582bb9404b0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc527cc740 10 client.0 _ll_register_callbacks cb 0x7fbc38004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.380+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47130/0 (socket says 172.21.15.7:47130) 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 -- 172.21.15.7:0/1111350535 learned_addr learned my addr 172.21.15.7:0/1111350535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc4da00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb948ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc4da00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb9443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 -- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 msgr2=0x5582bb9443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb9443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 -- 172.21.15.7:0/1111350535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 msgr2=0x5582bb948ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb948ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 -- 172.21.15.7:0/1111350535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bb6787d0 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc4da00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb948ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc47e00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc38032e60 tx=0x7fbc38032870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb871e9ed9d0403 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.381+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38059060 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbc38034650 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc400046d0 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbc38044040 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5582bb9b68c0 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc400046d0 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fbc38003870 con 0x5582bb9378d0 2024-04-20T21:34:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 10 client.15402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.382+0000 7fbc527cc740 20 client.15402 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb9b7040 0x5582bb9b9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc527cc740 4 client.15402 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582bb8a38e0 con 0x5582bb9b7040 2024-04-20T21:34:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc4da00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb9b7040 0x5582bb9b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc4da00640 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb9b7040 0x5582bb9b9460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a21fabfd122c5e6 server_cookie=cdc00d0840323fd9 in_seq=0 out_seq=0 2024-04-20T21:34:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.383+0000 7fbc2aa00640 10 client.15402 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc2aa00640 1 -- 172.21.15.7:0/1111350535 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582bb8a38e0 con 0x5582bb9b7040 2024-04-20T21:34:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc2aa00640 10 client.15402 handle_command_reply: tid=0 2024-04-20T21:34:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 client.15402 shutdown 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 msgr2=0x5582bb943e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc38032e60 tx=0x7fbc38032870 comp rx=0 tx=0).stop 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 shutdown_connections 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5582bb9b7040 0x5582bb9b9460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582bb9404b0 0x5582bb9443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5582bb9378d0 0x5582bb943e50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 --2- 172.21.15.7:0/1111350535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582bb936fb0 0x5582bb948ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc2aa00640 0 client.15402 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 >> 172.21.15.7:0/1111350535 conn(0x5582bb9316e0 msgr2=0x5582bb932830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 shutdown_connections 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 1 -- 172.21.15.7:0/1111350535 wait complete. 2024-04-20T21:34:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.384+0000 7fbc527cc740 20 client.15402 trim_cache size 0 max 16384 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 msgr2=0x7fbc300a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 0x7fbc300a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c0ce180 tx=0x7fbc3c0d4690 comp rx=0 tx=0).stop 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 msgr2=0x7fbc481a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 0x7fbc481a0740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e790 tx=0x7fbc3c00ec50 comp rx=0 tx=0).stop 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 shutdown_connections 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbc300a3ed0 0x7fbc300a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc48106710 0x7fbc4819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc48105d70 0x7fbc481a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 --2- 172.21.15.7:0/4138027222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc48104490 0x7fbc481a0740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 >> 172.21.15.7:0/4138027222 conn(0x7fbc480ffbf0 msgr2=0x7fbc48100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 shutdown_connections 2024-04-20T21:34:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.385+0000 7fbc4fa00640 1 -- 172.21.15.7:0/4138027222 wait complete. 2024-04-20T21:34:02.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:02 smithi007 ceph-mon[16867]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:02.399 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:02 smithi007 ceph-mon[16867]: mgrmap e26: x(active, since 40s), standbys: y, z 2024-04-20T21:34:02.400 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:02 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/190950378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:02.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:34:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:02 smithi016 ceph-mon[21409]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:02 smithi016 ceph-mon[21409]: mgrmap e26: x(active, since 40s), standbys: y, z 2024-04-20T21:34:02.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:02 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/190950378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:02.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:02 smithi186 ceph-mon[22157]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:02 smithi186 ceph-mon[22157]: mgrmap e26: x(active, since 40s), standbys: y, z 2024-04-20T21:34:02.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:02 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/190950378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 -- 172.21.15.7:0/792177276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8501041c0 msgr2=0x7fd8501045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 --2- 172.21.15.7:0/792177276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8501041c0 0x7fd8501045a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd840009900 tx=0x7fd84002bc50 comp rx=0 tx=0).stop 2024-04-20T21:34:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 -- 172.21.15.7:0/792177276 shutdown_connections 2024-04-20T21:34:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 --2- 172.21.15.7:0/792177276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd850109290 0x7fd850110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 --2- 172.21.15.7:0/792177276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd850104b70 0x7fd850108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 --2- 172.21.15.7:0/792177276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8501041c0 0x7fd8501045a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 -- 172.21.15.7:0/792177276 >> 172.21.15.7:0/792177276 conn(0x7fd8500ffa00 msgr2=0x7fd850101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 -- 172.21.15.7:0/792177276 shutdown_connections 2024-04-20T21:34:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.617+0000 7fd84f200640 1 -- 172.21.15.7:0/792177276 wait complete. 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.618+0000 7fd84f200640 1 Processor -- start 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.618+0000 7fd84f200640 1 -- start start 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd850109290 0x7fd8501a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd85019a6d0 0x7fd85019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd850114390 con 0x7fd8501041c0 2024-04-20T21:34:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd850114210 con 0x7fd850109290 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd850114510 con 0x7fd85019a6d0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47150/0 (socket says 172.21.15.7:47150) 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84de00640 1 -- 172.21.15.7:0/620814446 learned_addr learned my addr 172.21.15.7:0/620814446 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84e800640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd85019a6d0 0x7fd85019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.619+0000 7fd84d400640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd850109290 0x7fd8501a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 -- 172.21.15.7:0/620814446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd85019a6d0 msgr2=0x7fd85019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd85019a6d0 0x7fd85019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 -- 172.21.15.7:0/620814446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd850109290 msgr2=0x7fd8501a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd850109290 0x7fd8501a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 -- 172.21.15.7:0/620814446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85019b2d0 con 0x7fd8501041c0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84de00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd840005ab0 tx=0x7fd8400042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd846a00640 1 -- 172.21.15.7:0/620814446 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840041070 con 0x7fd8501041c0 2024-04-20T21:34:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.620+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85019b560 con 0x7fd8501041c0 2024-04-20T21:34:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.621+0000 7fd846a00640 1 -- 172.21.15.7:0/620814446 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8400044c0 con 0x7fd8501041c0 2024-04-20T21:34:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.621+0000 7fd846a00640 1 -- 172.21.15.7:0/620814446 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840003d80 con 0x7fd8501041c0 2024-04-20T21:34:02.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.621+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8501a7750 con 0x7fd8501041c0 2024-04-20T21:34:02.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.624+0000 7fd857d8f740 1 Processor -- start 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.624+0000 7fd846a00640 1 -- 172.21.15.7:0/620814446 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fd840044070 con 0x7fd8501041c0 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.624+0000 7fd857d8f740 1 -- start start 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.624+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.624+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd857d8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560644555050 con 0x5606445f0170 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd857d8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5606444fb490 con 0x5606445f9670 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd857d8f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560644551010 con 0x5606445f0a90 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47158/0 (socket says 172.21.15.7:47158) 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 -- 172.21.15.7:0/1649966290 learned_addr learned my addr 172.21.15.7:0/1649966290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84d400640 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 -- 172.21.15.7:0/1649966290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 msgr2=0x5606445f9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 -- 172.21.15.7:0/1649966290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 msgr2=0x5606445fba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445fba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84e800640 1 -- 172.21.15.7:0/1649966290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560644503070 con 0x5606445f0170 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.625+0000 7fd84d400640 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.626+0000 7fd84e800640 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd848009f50 tx=0x7fd8480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad56415fbe658318 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.626+0000 7fd83b400640 1 -- 172.21.15.7:0/1649966290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848004d20 con 0x5606445f0170 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.626+0000 7fd83b400640 1 -- 172.21.15.7:0/1649966290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd848004050 con 0x5606445f0170 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.626+0000 7fd83b400640 1 -- 172.21.15.7:0/1649966290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848005280 con 0x5606445f0170 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.626+0000 7fd846a00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 0x7fd8340a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd846a00640 1 -- 172.21.15.7:0/620814446 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd8400f22a0 con 0x7fd8501041c0 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd84de00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 0x7fd8340a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 -- 172.21.15.7:0/1649966290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 msgr2=0x5606445f0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd848009f50 tx=0x7fd8480118d0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 -- 172.21.15.7:0/1649966290 shutdown_connections 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445fba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1649966290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f0550 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 -- 172.21.15.7:0/1649966290 >> 172.21.15.7:0/1649966290 conn(0x5606445ea8a0 msgr2=0x5606445eac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 -- 172.21.15.7:0/1649966290 shutdown_connections 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd857d8f740 1 -- 172.21.15.7:0/1649966290 wait complete. 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.627+0000 7fd84de00640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 0x7fd8340a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8400ef970 tx=0x7fd8400f5d50 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 -- start start 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 Processor -- start 2024-04-20T21:34:02.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.628+0000 7fd857d8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5606445fc460 con 0x5606445f0170 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd857d8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56064450e860 con 0x5606445f9670 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd857d8f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5606445fc5e0 con 0x5606445f0a90 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd857d8f740 10 client.0 _ll_register_callbacks cb 0x5606444dcfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47164/0 (socket says 172.21.15.7:47164) 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84e800640 1 -- 172.21.15.7:0/3506329047 learned_addr learned my addr 172.21.15.7:0/3506329047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84e800640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 -- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 msgr2=0x5606445f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 -- 172.21.15.7:0/3506329047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 msgr2=0x5606445f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.629+0000 7fd84d400640 1 -- 172.21.15.7:0/3506329047 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560644512950 con 0x5606445f0a90 2024-04-20T21:34:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd84d400640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f6c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd85006b4d0 tx=0x7fd83c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2bcec8777a90de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd82fe00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c004030 con 0x5606445f0a90 2024-04-20T21:34:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd848004050 con 0x5606445f0a90 2024-04-20T21:34:02.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd83c0041d0 con 0x5606445f0a90 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd83c0043d0 con 0x5606445f0a90 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd857d8f740 10 client.25525 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.630+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5606444e0820 con 0x5606445f0a90 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd848004050 con 0x5606445f0a90 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fd83c003b90 con 0x5606445f0a90 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 10 client.25525 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 10 client.25525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 20 client.25525 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560644609690 0x56064460ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 4 client.25525 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.631+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560644562a70 con 0x560644609690 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.632+0000 7fd84e800640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560644609690 0x56064460ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.632+0000 7fd84e800640 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560644609690 0x56064460ba80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff3b953f44d047d server_cookie=f8a5f9e3ceddb918 in_seq=0 out_seq=0 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.632+0000 7fd82fe00640 10 client.25525 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd82fe00640 1 -- 172.21.15.7:0/3506329047 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560644562a70 con 0x560644609690 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd82fe00640 10 client.25525 handle_command_reply: tid=0 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 client.25525 shutdown 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 msgr2=0x5606445f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f6c10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd85006b4d0 tx=0x7fd83c012b60 comp rx=0 tx=0).stop 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 shutdown_connections 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560644609690 0x56064460ba80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0a90 0x5606445f6c10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 --2- 172.21.15.7:0/3506329047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 >> 172.21.15.7:0/3506329047 conn(0x5606445ea8a0 msgr2=0x5606445ec5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd82fe00640 0 client.25525 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 shutdown_connections 2024-04-20T21:34:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.633+0000 7fd857d8f740 1 -- 172.21.15.7:0/3506329047 wait complete. 2024-04-20T21:34:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.634+0000 7fd857d8f740 20 client.25525 trim_cache size 0 max 16384 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.648+0000 7fd857d8f740 1 Processor -- start 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.648+0000 7fd857d8f740 1 -- start start 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0170 0x5606445f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501b2550 con 0x5606445f0a90 2024-04-20T21:34:02.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501afe10 con 0x5606445f9670 2024-04-20T21:34:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd857d8f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501adc80 con 0x5606445f0170 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47180/0 (socket says 172.21.15.7:47180) 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 -- 172.21.15.7:0/2176308984 learned_addr learned my addr 172.21.15.7:0/2176308984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84e800640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0170 0x5606445f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84e800640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 -- 172.21.15.7:0/2176308984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0170 msgr2=0x5606445f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0170 0x5606445f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 -- 172.21.15.7:0/2176308984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 msgr2=0x5606445f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84d400640 1 -- 172.21.15.7:0/2176308984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606442673f0 con 0x5606445f0a90 2024-04-20T21:34:02.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.649+0000 7fd84e800640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.650+0000 7fd84d400640 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009fb0 tx=0x7fd83c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f73431372dd9da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.650+0000 7fd83aa00640 1 -- 172.21.15.7:0/2176308984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c003fe0 con 0x5606445f0a90 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.650+0000 7fd83aa00640 1 -- 172.21.15.7:0/2176308984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c00fe00 con 0x5606445f0a90 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.650+0000 7fd83aa00640 1 -- 172.21.15.7:0/2176308984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c026640 con 0x5606445f0a90 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- 172.21.15.7:0/2176308984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 msgr2=0x560644605320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009fb0 tx=0x7fd83c009620 comp rx=0 tx=0).stop 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- 172.21.15.7:0/2176308984 shutdown_connections 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f9670 0x5606445f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0a90 0x560644605320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 --2- 172.21.15.7:0/2176308984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f0170 0x5606445f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- 172.21.15.7:0/2176308984 >> 172.21.15.7:0/2176308984 conn(0x5606445ea8a0 msgr2=0x5606445f6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- 172.21.15.7:0/2176308984 shutdown_connections 2024-04-20T21:34:02.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- 172.21.15.7:0/2176308984 wait complete. 2024-04-20T21:34:02.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 -- start start 2024-04-20T21:34:02.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.651+0000 7fd857d8f740 1 Processor -- start 2024-04-20T21:34:02.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f0a90 0x5606445fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f9670 0x56064466ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501ae2d0 con 0x5606445f0170 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501af740 con 0x5606445f0a90 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd8501ae9a0 con 0x5606445f9670 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd857d8f740 10 client.0 _ll_register_callbacks cb 0x7fd83c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd84e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47186/0 (socket says 172.21.15.7:47186) 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd84e800640 1 -- 172.21.15.7:0/1015310739 learned_addr learned my addr 172.21.15.7:0/1015310739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd84d400640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f0a90 0x5606445fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.652+0000 7fd84e800640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f9670 0x56064466ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 -- 172.21.15.7:0/1015310739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f9670 msgr2=0x56064466ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f9670 0x56064466ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 -- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f0a90 msgr2=0x5606445fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:02.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f0a90 0x5606445fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 -- 172.21.15.7:0/1015310739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606443317d0 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd84e800640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd848011dd0 tx=0x7fd84802b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496d15340b630ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd82fe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848003c70 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd848004050 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c00fe00 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.653+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd848003e10 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x560644670150 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd83c00fe00 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fd848035020 con 0x5606445f0170 2024-04-20T21:34:02.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 20 client.15420 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5606446706e0 0x560644672ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 4 client.15420 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56064455c8e0 con 0x5606446706e0 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.654+0000 7fd84d400640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5606446706e0 0x560644672ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd84d400640 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5606446706e0 0x560644672ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625fe56b822000ba server_cookie=7a65816a5292ae14 in_seq=0 out_seq=0 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd82fe00640 10 client.15420 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd82fe00640 1 -- 172.21.15.7:0/1015310739 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56064455c8e0 con 0x5606446706e0 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd82fe00640 10 client.15420 handle_command_reply: tid=0 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 client.15420 shutdown 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 msgr2=0x5606445fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd848011dd0 tx=0x7fd84802b040 comp rx=0 tx=0).stop 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 shutdown_connections 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5606445f0a90 0x5606445fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5606446706e0 0x560644672ad0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5606445f9670 0x56064466ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.655+0000 7fd857d8f740 1 --2- 172.21.15.7:0/1015310739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5606445f0170 0x5606445fd620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd82fe00640 0 client.15420 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 >> 172.21.15.7:0/1015310739 conn(0x5606445ea8a0 msgr2=0x5606445e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 shutdown_connections 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd857d8f740 1 -- 172.21.15.7:0/1015310739 wait complete. 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd857d8f740 20 client.15420 trim_cache size 0 max 16384 2024-04-20T21:34:02.659 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:02.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 msgr2=0x7fd8340a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 0x7fd8340a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8400ef970 tx=0x7fd8400f5d50 comp rx=0 tx=0).stop 2024-04-20T21:34:02.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 msgr2=0x7fd8501a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:02.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.656+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd840005ab0 tx=0x7fd8400042d0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 shutdown_connections 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd8340a3ed0 0x7fd8340a6390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd850109290 0x7fd8501a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd85019a6d0 0x7fd85019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 --2- 172.21.15.7:0/620814446 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8501041c0 0x7fd8501a0610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 >> 172.21.15.7:0/620814446 conn(0x7fd8500ffa00 msgr2=0x7fd850101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 shutdown_connections 2024-04-20T21:34:02.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:02.657+0000 7fd84f200640 1 -- 172.21.15.7:0/620814446 wait complete. 2024-04-20T21:34:03.675 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:34:03.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:34:03.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:34:03.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 -- 172.21.15.7:0/3150004793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa8109290 msgr2=0x7f5fa8110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 --2- 172.21.15.7:0/3150004793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa8109290 0x7f5fa8110e60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009be0 tx=0x7f5fa002bef0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 -- 172.21.15.7:0/3150004793 shutdown_connections 2024-04-20T21:34:03.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 --2- 172.21.15.7:0/3150004793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa8109290 0x7f5fa8110e60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 --2- 172.21.15.7:0/3150004793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8104b70 0x7f5fa8108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 --2- 172.21.15.7:0/3150004793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 -- 172.21.15.7:0/3150004793 >> 172.21.15.7:0/3150004793 conn(0x7f5fa80ffa00 msgr2=0x7f5fa8101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 -- 172.21.15.7:0/3150004793 shutdown_connections 2024-04-20T21:34:03.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.907+0000 7f5faca00640 1 -- 172.21.15.7:0/3150004793 wait complete. 2024-04-20T21:34:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.908+0000 7f5faca00640 1 Processor -- start 2024-04-20T21:34:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.908+0000 7f5faca00640 1 -- start start 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8109290 0x7f5fa81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8114380 con 0x7f5fa8109290 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8114200 con 0x7f5fa81041c0 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5faca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8114500 con 0x7f5fa819a640 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48450/0 (socket says 172.21.15.7:48450) 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa7400640 1 -- 172.21.15.7:0/2714092873 learned_addr learned my addr 172.21.15.7:0/2714092873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49182/0 (socket says 172.21.15.7:49182) 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.909+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8109290 0x7f5fa81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 -- 172.21.15.7:0/2714092873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 msgr2=0x7f5fa81a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 -- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8109290 msgr2=0x7f5fa81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8109290 0x7f5fa81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 -- 172.21.15.7:0/2714092873 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa819b240 con 0x7f5fa819a640 2024-04-20T21:34:03.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.910+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c430 tx=0x7f5fa00336b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.911+0000 7f5f9fe00640 1 -- 172.21.15.7:0/2714092873 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0043020 con 0x7f5fa819a640 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.911+0000 7f5f9fe00640 1 -- 172.21.15.7:0/2714092873 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0003e50 con 0x7f5fa819a640 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.911+0000 7f5f9fe00640 1 -- 172.21.15.7:0/2714092873 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa003e6b0 con 0x7f5fa819a640 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.911+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa819b4d0 con 0x7f5fa819a640 2024-04-20T21:34:03.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.911+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa81a7620 con 0x7f5fa819a640 2024-04-20T21:34:03.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.914+0000 7f5fb0e38740 1 Processor -- start 2024-04-20T21:34:03.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.914+0000 7f5f9fe00640 1 -- 172.21.15.7:0/2714092873 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f5fa0046020 con 0x7f5fa819a640 2024-04-20T21:34:03.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.914+0000 7f5fb0e38740 1 -- start start 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562299e1fd20 con 0x562299ebb0f0 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562299dc63e0 con 0x562299ebba10 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562299e1bce0 con 0x562299ec45f0 2024-04-20T21:34:03.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60408/0 (socket says 172.21.15.7:60408) 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 -- 172.21.15.7:0/1237443871 learned_addr learned my addr 172.21.15.7:0/1237443871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 -- 172.21.15.7:0/1237443871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 msgr2=0x562299ec69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 -- 172.21.15.7:0/1237443871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 msgr2=0x562299ec40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.915+0000 7f5fa6a00640 1 -- 172.21.15.7:0/1237443871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562299de5980 con 0x562299ebb0f0 2024-04-20T21:34:03.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f940134d0 tx=0x7f5f940097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f5aa842d48e94f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5f9fe00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 0x7f5f8c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5f93e00640 1 -- 172.21.15.7:0/1237443871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004020 con 0x562299ebb0f0 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5f93e00640 1 -- 172.21.15.7:0/1237443871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f94005500 con 0x562299ebb0f0 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5f93e00640 1 -- 172.21.15.7:0/1237443871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94004c60 con 0x562299ebb0f0 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5f9fe00640 1 -- 172.21.15.7:0/2714092873 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5fa00f2e30 con 0x7f5fa819a640 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.916+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 0x7f5f8c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1237443871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 msgr2=0x562299ebb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f940134d0 tx=0x7f5f940097b0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1237443871 shutdown_connections 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1237443871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ebb4d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1237443871 >> 172.21.15.7:0/1237443871 conn(0x562299eb5820 msgr2=0x562299eb5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1237443871 shutdown_connections 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1237443871 wait complete. 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.917+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 0x7f5f8c0a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009620 tx=0x7f5f98005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 -- start start 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 Processor -- start 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562299ec73e0 con 0x562299ebb0f0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562299dd97a0 con 0x562299ebba10 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.918+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562299ec7560 con 0x562299ec45f0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fb0e38740 10 client.0 _ll_register_callbacks cb 0x562299ddbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60424/0 (socket says 172.21.15.7:60424) 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 -- 172.21.15.7:0/1318087125 learned_addr learned my addr 172.21.15.7:0/1318087125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 -- 172.21.15.7:0/1318087125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 msgr2=0x562299ec2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 -- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 msgr2=0x562299ec1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.919+0000 7f5fa7400640 1 -- 172.21.15.7:0/1318087125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562299bfa7e0 con 0x562299ebb0f0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800f910 tx=0x7f5f98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a6d804ccc81b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5f90c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98011bd0 con 0x562299ebb0f0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5f98011d70 con 0x562299ebb0f0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f94005500 con 0x562299ebb0f0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5f98029440 con 0x562299ebb0f0 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5fb0e38740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562299c5af70 con 0x562299ebb0f0 2024-04-20T21:34:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.920+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f94005500 con 0x562299ebb0f0 2024-04-20T21:34:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f5f98027070 con 0x562299ebb0f0 2024-04-20T21:34:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:03.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 20 client.15432 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299ed4a80 0x562299ed6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 4 client.15432 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562299e2d740 con 0x562299ed4a80 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.921+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299ed4a80 0x562299ed6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.922+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299ed4a80 0x562299ed6f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e5d225dcfa3f19 server_cookie=3466b4dcb43e17ef in_seq=0 out_seq=0 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.922+0000 7f5f90c00640 10 client.15432 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5f90c00640 1 -- 172.21.15.7:0/1318087125 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562299e2d740 con 0x562299ed4a80 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5f90c00640 10 client.15432 handle_command_reply: tid=0 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 client.15432 shutdown 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 msgr2=0x562299ec1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800f910 tx=0x7f5f98004730 comp rx=0 tx=0).stop 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 shutdown_connections 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ec1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299ed4a80 0x562299ed6f40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1318087125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec1690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 >> 172.21.15.7:0/1318087125 conn(0x562299eb5820 msgr2=0x562299eb7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5f90c00640 0 client.15432 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 shutdown_connections 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1318087125 wait complete. 2024-04-20T21:34:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.923+0000 7f5fb0e38740 20 client.15432 trim_cache size 0 max 16384 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 Processor -- start 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 -- start start 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ed02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299eb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81b2460 con 0x562299ebb0f0 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81afd20 con 0x562299ebba10 2024-04-20T21:34:03.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81adb90 con 0x562299ec45f0 2024-04-20T21:34:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fa7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60438/0 (socket says 172.21.15.7:60438) 2024-04-20T21:34:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.943+0000 7f5fa7400640 1 -- 172.21.15.7:0/2026968694 learned_addr learned my addr 172.21.15.7:0/2026968694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299eb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ed02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 -- 172.21.15.7:0/2026968694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 msgr2=0x562299eb7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299eb7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 -- 172.21.15.7:0/2026968694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 msgr2=0x562299ed02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ed02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 -- 172.21.15.7:0/2026968694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562299af2170 con 0x562299ebb0f0 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ed02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:03.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5fa7400640 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f980097f0 tx=0x7f5f9800ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0901049622fa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5f93400640 1 -- 172.21.15.7:0/2026968694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98036030 con 0x562299ebb0f0 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5f93400640 1 -- 172.21.15.7:0/2026968694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f98005500 con 0x562299ebb0f0 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.944+0000 7f5f93400640 1 -- 172.21.15.7:0/2026968694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f980047a0 con 0x562299ebb0f0 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 -- 172.21.15.7:0/2026968694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 msgr2=0x562299ec3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5f980097f0 tx=0x7f5f9800ff90 comp rx=0 tx=0).stop 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 -- 172.21.15.7:0/2026968694 shutdown_connections 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebba10 0x562299ed02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299eb7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/2026968694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebb0f0 0x562299ec3dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 -- 172.21.15.7:0/2026968694 >> 172.21.15.7:0/2026968694 conn(0x562299eb5820 msgr2=0x562299eb6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 -- 172.21.15.7:0/2026968694 shutdown_connections 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.945+0000 7f5fb0e38740 1 -- 172.21.15.7:0/2026968694 wait complete. 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 -- start start 2024-04-20T21:34:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 Processor -- start 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebb0f0 0x562299eccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 0x562299ec7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81ae1e0 con 0x562299ebba10 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81af650 con 0x562299ebb0f0 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa81ae8b0 con 0x562299ec45f0 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fb0e38740 10 client.0 _ll_register_callbacks cb 0x7f5f98012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.946+0000 7f5fa6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 0x562299ec7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49222/0 (socket says 172.21.15.7:49222) 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 -- 172.21.15.7:0/1187692667 learned_addr learned my addr 172.21.15.7:0/1187692667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebb0f0 0x562299eccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 -- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebb0f0 msgr2=0x562299eccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebb0f0 0x562299eccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 -- 172.21.15.7:0/1187692667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 msgr2=0x562299ec7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 0x562299ec7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 -- 172.21.15.7:0/1187692667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562299ec8c90 con 0x562299ec45f0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa6a00640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 0x562299ec7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5fa7e00640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00bb150 tx=0x7f5fa0033a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539fda63f8f7a948 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5f90c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.947+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00df2c0 con 0x562299ec45f0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5fa0004b40 con 0x562299ec45f0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f98005500 con 0x562299ec45f0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5fa00becf0 con 0x562299ec45f0 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 10 client.25549 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562299f3b170 con 0x562299ec45f0 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f98005500 con 0x562299ec45f0 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f5fa00be140 con 0x562299ec45f0 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 10 client.25549 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 20 client.25549 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299f3b840 0x562299f3dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 4 client.25549 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.948+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562299e275b0 con 0x562299f3b840 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.949+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299f3b840 0x562299f3dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.949+0000 7f5fa7400640 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299f3b840 0x562299f3dc60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced0cbca20a6ab05 server_cookie=dedeb167612ab384 in_seq=0 out_seq=0 2024-04-20T21:34:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.949+0000 7f5f90c00640 10 client.25549 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.950+0000 7f5f90c00640 1 -- 172.21.15.7:0/1187692667 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562299e275b0 con 0x562299f3b840 2024-04-20T21:34:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.950+0000 7f5f90c00640 10 client.25549 handle_command_reply: tid=0 2024-04-20T21:34:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.950+0000 7f5fb0e38740 1 client.25549 shutdown 2024-04-20T21:34:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 msgr2=0x562299ec8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00bb150 tx=0x7f5fa0033a30 comp rx=0 tx=0).stop 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 shutdown_connections 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x562299f3b840 0x562299f3dc60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562299ec45f0 0x562299ec8460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562299ebba10 0x562299ec7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 --2- 172.21.15.7:0/1187692667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562299ebb0f0 0x562299eccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 >> 172.21.15.7:0/1187692667 conn(0x562299eb5820 msgr2=0x562299eb4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5f90c00640 0 client.25549 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 shutdown_connections 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 1 -- 172.21.15.7:0/1187692667 wait complete. 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.951+0000 7f5fb0e38740 20 client.25549 trim_cache size 0 max 16384 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:03.952 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:34:03.953 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f1c38fc7-f095-4e58-a7a5-2a3cebfd3934", 2024-04-20T21:34:03.953 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:34:03.953 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:03.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 msgr2=0x7f5f8c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 0x7f5f8c0a6390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009620 tx=0x7f5f98005f90 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 msgr2=0x7f5fa819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5fa002c430 tx=0x7f5fa00336b0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 shutdown_connections 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fa819a640 0x7f5fa819aac0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fa8109290 0x7f5fa81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5f8c0a3ed0 0x7f5f8c0a6390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 --2- 172.21.15.7:0/2714092873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fa81041c0 0x7f5fa81a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 >> 172.21.15.7:0/2714092873 conn(0x7f5fa80ffa00 msgr2=0x7f5fa81004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 shutdown_connections 2024-04-20T21:34:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:03.952+0000 7f5faca00640 1 -- 172.21.15.7:0/2714092873 wait complete. 2024-04-20T21:34:03.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:03.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 -- 172.21.15.7:0/2827648215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074390 msgr2=0x7f3e40074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 --2- 172.21.15.7:0/2827648215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074390 0x7f3e40074770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009ca0 tx=0x7f3e3002c190 comp rx=0 tx=0).stop 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 -- 172.21.15.7:0/2827648215 shutdown_connections 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 --2- 172.21.15.7:0/2827648215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e4010e570 0x7f3e401109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 --2- 172.21.15.7:0/2827648215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e40074cb0 0x7f3e4010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 --2- 172.21.15.7:0/2827648215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074390 0x7f3e40074770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 -- 172.21.15.7:0/2827648215 >> 172.21.15.7:0/2827648215 conn(0x7f3e400fd840 msgr2=0x7f3e400ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.185+0000 7f3e44c00640 1 -- 172.21.15.7:0/2827648215 shutdown_connections 2024-04-20T21:34:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.186+0000 7f3e44c00640 1 -- 172.21.15.7:0/2827648215 wait complete. 2024-04-20T21:34:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.186+0000 7f3e44c00640 1 Processor -- start 2024-04-20T21:34:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 -- start start 2024-04-20T21:34:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 0x7f3e4010dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 0x7f3e40105170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40103820 con 0x7f3e4010e570 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401036a0 con 0x7f3e40074cb0 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e44c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401039a0 con 0x7f3e40074390 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 0x7f3e4010dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48502/0 (socket says 172.21.15.7:48502) 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e3ea00640 1 -- 172.21.15.7:0/4232284560 learned_addr learned my addr 172.21.15.7:0/4232284560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.187+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 0x7f3e40105170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 -- 172.21.15.7:0/4232284560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 msgr2=0x7f3e4010dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 0x7f3e4010dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 -- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 msgr2=0x7f3e40105170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 0x7f3e40105170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 -- 172.21.15.7:0/4232284560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e401058a0 con 0x7f3e40074cb0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 0x7f3e40105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3f400640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 0x7f3e4010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9f0 tx=0x7f3e2c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e37e00640 1 -- 172.21.15.7:0/4232284560 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00cd80 con 0x7f3e40074cb0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e401096c0 con 0x7f3e40074cb0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e37e00640 1 -- 172.21.15.7:0/4232284560 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e2c010910 con 0x7f3e40074cb0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e37e00640 1 -- 172.21.15.7:0/4232284560 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c018990 con 0x7f3e40074cb0 2024-04-20T21:34:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.188+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e40109c10 con 0x7f3e40074cb0 2024-04-20T21:34:04.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.192+0000 7f3e37e00640 1 -- 172.21.15.7:0/4232284560 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f3e2c010ac0 con 0x7f3e40074cb0 2024-04-20T21:34:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.192+0000 7f3e49004740 1 Processor -- start 2024-04-20T21:34:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.192+0000 7f3e49004740 1 -- start start 2024-04-20T21:34:04.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.192+0000 7f3e49004740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.192+0000 7f3e49004740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e49004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568071a3880 0x5568071a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e49004740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556807100150 con 0x55680719a380 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e49004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568070a6370 con 0x5568071a3880 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e49004740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5568070fc110 con 0x55680719aca0 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60468/0 (socket says 172.21.15.7:60468) 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 -- 172.21.15.7:0/1362423670 learned_addr learned my addr 172.21.15.7:0/1362423670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568071a3880 0x5568071a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3f400640 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 -- 172.21.15.7:0/1362423670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 msgr2=0x5568071a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e37e00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 0x7f3e240a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 -- 172.21.15.7:0/1362423670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568071a3880 msgr2=0x5568071a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568071a3880 0x5568071a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.193+0000 7f3e3fe00640 1 -- 172.21.15.7:0/1362423670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556806edc7d0 con 0x55680719a380 2024-04-20T21:34:04.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.194+0000 7f3e37e00640 1 -- 172.21.15.7:0/4232284560 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3e2c0d0e70 con 0x7f3e40074cb0 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.194+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 0x7f3e240a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.194+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009c20 tx=0x7f3e380117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cf05bd93ca5785 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.194+0000 7f3e2be00640 1 -- 172.21.15.7:0/1362423670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38012650 con 0x55680719a380 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e2be00640 1 -- 172.21.15.7:0/1362423670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38012b40 con 0x55680719a380 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e2be00640 1 -- 172.21.15.7:0/1362423670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800f440 con 0x55680719a380 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 -- 172.21.15.7:0/1362423670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 msgr2=0x55680719a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 0x7f3e240a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009fd0 tx=0x7f3e38021000 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009c20 tx=0x7f3e380117b0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 -- 172.21.15.7:0/1362423670 shutdown_connections 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5568071a3880 0x5568071a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 --2- 172.21.15.7:0/1362423670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55680719a380 0x55680719a760 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.195+0000 7f3e49004740 1 -- 172.21.15.7:0/1362423670 >> 172.21.15.7:0/1362423670 conn(0x556807194ab0 msgr2=0x556807194e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.196+0000 7f3e49004740 1 -- 172.21.15.7:0/1362423670 shutdown_connections 2024-04-20T21:34:04.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.196+0000 7f3e49004740 1 -- 172.21.15.7:0/1362423670 wait complete. 2024-04-20T21:34:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.196+0000 7f3e49004740 1 -- start start 2024-04-20T21:34:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.196+0000 7f3e49004740 1 Processor -- start 2024-04-20T21:34:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071a1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568071a6670 con 0x5568071a3880 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5568070b9740 con 0x55680719a380 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5568071a67f0 con 0x55680719aca0 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e49004740 10 client.0 _ll_register_callbacks cb 0x5568070b4b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e3ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49272/0 (socket says 172.21.15.7:49272) 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e3ea00640 1 -- 172.21.15.7:0/1771311268 learned_addr learned my addr 172.21.15.7:0/1771311268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.197+0000 7f3e3f400640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 -- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 msgr2=0x5568071a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 -- 172.21.15.7:0/1771311268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 msgr2=0x5568071a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 -- 172.21.15.7:0/1771311268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556806e123f0 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9c0 tx=0x7f3e2c09c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d5ac5f92fc1be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e28c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c018d30 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e38012b40 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3e2c09c070 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3e2c09d320 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e49004740 10 client.25561 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.198+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556806dd20f0 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38012b40 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f3e2c098970 con 0x55680719aca0 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 10 client.25561 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:04.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 20 client.25561 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5568071b3c00 0x5568071b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 4 client.25561 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55680710db70 con 0x5568071b3c00 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.199+0000 7f3e3f400640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5568071b3c00 0x5568071b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.200+0000 7f3e3f400640 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5568071b3c00 0x5568071b6010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c584ef88204aec server_cookie=dc6c07997f213175 in_seq=0 out_seq=0 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.200+0000 7f3e28c00640 10 client.25561 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e28c00640 1 -- 172.21.15.7:0/1771311268 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55680710db70 con 0x5568071b3c00 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e28c00640 10 client.25561 handle_command_reply: tid=0 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 client.25561 shutdown 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 msgr2=0x5568071a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9c0 tx=0x7f3e2c09c9f0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 shutdown_connections 2024-04-20T21:34:04.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5568071b3c00 0x5568071b6010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x5568071a0e50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e28c00640 0 client.25561 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 --2- 172.21.15.7:0/1771311268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 >> 172.21.15.7:0/1771311268 conn(0x556807194ab0 msgr2=0x556807196800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.201+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 shutdown_connections 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.202+0000 7f3e49004740 1 -- 172.21.15.7:0/1771311268 wait complete. 2024-04-20T21:34:04.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.202+0000 7f3e49004740 20 client.25561 trim_cache size 0 max 16384 2024-04-20T21:34:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.221+0000 7f3e49004740 1 Processor -- start 2024-04-20T21:34:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.221+0000 7f3e49004740 1 -- start start 2024-04-20T21:34:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401bf120 con 0x5568071a3880 2024-04-20T21:34:04.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401bc9e0 con 0x55680719a380 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e49004740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401ba830 con 0x55680719aca0 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48540/0 (socket says 172.21.15.7:48540) 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 -- 172.21.15.7:0/3884324677 learned_addr learned my addr 172.21.15.7:0/3884324677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 -- 172.21.15.7:0/3884324677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 msgr2=0x55680719f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 -- 172.21.15.7:0/3884324677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 msgr2=0x5568071af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.222+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.223+0000 7f3e3f400640 1 -- 172.21.15.7:0/3884324677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55680719aad0 con 0x55680719a380 2024-04-20T21:34:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.223+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e30004af0 tx=0x7f3e30005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ad82f5442ddf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.223+0000 7f3e2b400640 1 -- 172.21.15.7:0/3884324677 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30039040 con 0x55680719a380 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.223+0000 7f3e2b400640 1 -- 172.21.15.7:0/3884324677 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3004e070 con 0x55680719a380 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.223+0000 7f3e2b400640 1 -- 172.21.15.7:0/3884324677 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30034bf0 con 0x55680719a380 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- 172.21.15.7:0/3884324677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 msgr2=0x5568071a2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e30004af0 tx=0x7f3e30005fd0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- 172.21.15.7:0/3884324677 shutdown_connections 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x5568071af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 --2- 172.21.15.7:0/3884324677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x5568071a2fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- 172.21.15.7:0/3884324677 >> 172.21.15.7:0/3884324677 conn(0x556807194ab0 msgr2=0x556807193b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- 172.21.15.7:0/3884324677 shutdown_connections 2024-04-20T21:34:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- 172.21.15.7:0/3884324677 wait complete. 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.224+0000 7f3e49004740 1 -- start start 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 Processor -- start 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x55680719d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401baea0 con 0x5568071a3880 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401bc310 con 0x55680719a380 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401bb570 con 0x55680719aca0 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.225+0000 7f3e49004740 10 client.0 _ll_register_callbacks cb 0x7f3e2c0d4610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60484/0 (socket says 172.21.15.7:60484) 2024-04-20T21:34:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 -- 172.21.15.7:0/3834259289 learned_addr learned my addr 172.21.15.7:0/3834259289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x55680719d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 -- 172.21.15.7:0/3834259289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 msgr2=0x55680719ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 -- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 msgr2=0x55680719d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x55680719d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 -- 172.21.15.7:0/3834259289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568072193a0 con 0x5568071a3880 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3ea00640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e3fe00640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009c20 tx=0x7f3e38012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406af871dc0f2dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.226+0000 7f3e28c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38038070 con 0x5568071a3880 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3e38022040 con 0x5568071a3880 2024-04-20T21:34:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3004e070 con 0x5568071a3880 2024-04-20T21:34:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3e38043050 con 0x5568071a3880 2024-04-20T21:34:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e49004740 10 client.15456 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55680721a690 con 0x5568071a3880 2024-04-20T21:34:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.227+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3004e070 con 0x5568071a3880 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f3e3801c6a0 con 0x5568071a3880 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 10 client.15456 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 20 client.15456 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55680721ab20 0x55680721cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 4 client.15456 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568071079e0 con 0x55680721ab20 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.228+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55680721ab20 0x55680721cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e3f400640 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55680721ab20 0x55680721cf10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=577a476ddbe75020 server_cookie=b58bc527004cb9af in_seq=0 out_seq=0 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e28c00640 10 client.15456 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e28c00640 1 -- 172.21.15.7:0/3834259289 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5568071079e0 con 0x55680721ab20 2024-04-20T21:34:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e28c00640 10 client.15456 handle_command_reply: tid=0 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e49004740 1 client.15456 shutdown 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 msgr2=0x55680719e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.229+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009c20 tx=0x7f3e38012f60 comp rx=0 tx=0).stop 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 shutdown_connections 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55680721ab20 0x55680721cf10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568071a3880 0x55680719e300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55680719aca0 0x55680719ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 --2- 172.21.15.7:0/3834259289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55680719a380 0x55680719d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e28c00640 0 client.15456 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 >> 172.21.15.7:0/3834259289 conn(0x556807194ab0 msgr2=0x556807193280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 shutdown_connections 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 1 -- 172.21.15.7:0/3834259289 wait complete. 2024-04-20T21:34:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e49004740 20 client.15456 trim_cache size 0 max 16384 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "f1c38fc7-f095-4e58-a7a5-2a3cebfd3934": { 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "f1c38fc7-f095-4e58-a7a5-2a3cebfd3934", 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:34:04.233 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 msgr2=0x7f3e240a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 0x7f3e240a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009fd0 tx=0x7f3e38021000 comp rx=0 tx=0).stop 2024-04-20T21:34:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 msgr2=0x7f3e40104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.230+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e9f0 tx=0x7f3e2c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 shutdown_connections 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3e240a3ed0 0x7f3e240a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e40074cb0 0x7f3e40104c30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e4010e570 0x7f3e40105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 --2- 172.21.15.7:0/4232284560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e40074390 0x7f3e4010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 >> 172.21.15.7:0/4232284560 conn(0x7f3e400fd840 msgr2=0x7f3e400ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 shutdown_connections 2024-04-20T21:34:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:04.231+0000 7f3e44c00640 1 -- 172.21.15.7:0/4232284560 wait complete. 2024-04-20T21:34:04.248 INFO:tasks.cephfs.filesystem:scrub status for tag:f1c38fc7-f095-4e58-a7a5-2a3cebfd3934 - {'path': '/', 'tag': 'f1c38fc7-f095-4e58-a7a5-2a3cebfd3934', 'options': 'recursive,force'} 2024-04-20T21:34:04.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:04 smithi016 ceph-mon[21409]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:04.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:04 smithi186 ceph-mon[22157]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:04.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:04 smithi007 ceph-mon[16867]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:05.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:05 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:34:05.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:05 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:05.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:05 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:34:05.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:05 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:05.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:05 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:34:05.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:05 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:05.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:05 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:34:05.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:05 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:05.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:05 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:34:05.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:05 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:05.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:05 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:34:05.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:05 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:06.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:06 smithi016 ceph-mon[21409]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:06.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:06 smithi186 ceph-mon[22157]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:06.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:06 smithi007 ceph-mon[16867]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:07.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:07 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:34:07.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:07 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:34:07.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:07 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:34:08.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:08 smithi016 ceph-mon[21409]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-20T21:34:08.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:08 smithi186 ceph-mon[22157]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-20T21:34:08.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:08 smithi007 ceph-mon[16867]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-20T21:34:10.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:10 smithi016 ceph-mon[21409]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:10.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:10 smithi186 ceph-mon[22157]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:10.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:10 smithi007 ceph-mon[16867]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:34:12.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:12 smithi186 ceph-mon[22157]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:12.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:12 smithi007 ceph-mon[16867]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:12.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:12 smithi016 ceph-mon[21409]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-20T21:34:14.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:14 smithi186 ceph-mon[22157]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-20T21:34:14.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:14 smithi007 ceph-mon[16867]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-20T21:34:14.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:14 smithi016 ceph-mon[21409]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-20T21:34:16.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:16 smithi186 ceph-mon[22157]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-20T21:34:16.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:16 smithi007 ceph-mon[16867]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-20T21:34:16.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:16 smithi016 ceph-mon[21409]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-20T21:34:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:18 smithi186 ceph-mon[22157]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-20T21:34:18.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:18 smithi007 ceph-mon[16867]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-20T21:34:18.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:18 smithi016 ceph-mon[21409]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-20T21:34:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:20 smithi007 ceph-mon[16867]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-20T21:34:20.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:20 smithi016 ceph-mon[21409]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-20T21:34:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:20 smithi186 ceph-mon[22157]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-20T21:34:22.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:22 smithi007 ceph-mon[16867]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-20T21:34:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:22 smithi016 ceph-mon[21409]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-20T21:34:22.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:22 smithi186 ceph-mon[22157]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-20T21:34:24.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:24 smithi007 ceph-mon[16867]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-20T21:34:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:24 smithi016 ceph-mon[21409]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-20T21:34:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:24 smithi186 ceph-mon[22157]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-20T21:34:26.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:26 smithi007 ceph-mon[16867]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:26 smithi016 ceph-mon[21409]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:26 smithi186 ceph-mon[22157]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:28.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:28 smithi007 ceph-mon[16867]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:28 smithi016 ceph-mon[21409]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:28 smithi186 ceph-mon[22157]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:30.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:30 smithi007 ceph-mon[16867]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:34:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:30 smithi016 ceph-mon[21409]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:34:30.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:30 smithi186 ceph-mon[22157]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:34:31.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:31 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:31.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:31 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:31.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:31 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:34:32.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:32 smithi007 ceph-mon[16867]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:32 smithi016 ceph-mon[21409]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:32 smithi186 ceph-mon[22157]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:33 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:34:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:33 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:34:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:33 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:34:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:33 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:34:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:33 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:34:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:33 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:34:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:33 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:34:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:33 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:34:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:33 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:34:34.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:34.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3407093901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc810e620 msgr2=0x7f0bc8110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3407093901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc810e620 0x7f0bc8110a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009ca0 tx=0x7f0bc002c190 comp rx=0 tx=0).stop 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3407093901 shutdown_connections 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3407093901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc810e620 0x7f0bc8110a10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3407093901 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc8074c30 0x7f0bc810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3407093901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 0x7f0bc80746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3407093901 >> 172.21.15.7:0/3407093901 conn(0x7f0bc80fd8b0 msgr2=0x7f0bc80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3407093901 shutdown_connections 2024-04-20T21:34:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.445+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3407093901 wait complete. 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.446+0000 7f0bc7e00640 1 Processor -- start 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.446+0000 7f0bc7e00640 1 -- start start 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 0x7f0bc81a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc810e620 0x7f0bc819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8114300 con 0x7f0bc8074310 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8114180 con 0x7f0bc8074c30 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8114480 con 0x7f0bc810e620 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 0x7f0bc81a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45282/0 (socket says 172.21.15.7:45282) 2024-04-20T21:34:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc6000640 1 -- 172.21.15.7:0/3847037332 learned_addr learned my addr 172.21.15.7:0/3847037332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.447+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc810e620 0x7f0bc819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 -- 172.21.15.7:0/3847037332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc810e620 msgr2=0x7f0bc819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc810e620 0x7f0bc819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 -- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 msgr2=0x7f0bc81a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 0x7f0bc81a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 -- 172.21.15.7:0/3847037332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc819af40 con 0x7f0bc8074c30 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400ea60 tx=0x7f0bb400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.448+0000 7f0bbf400640 1 -- 172.21.15.7:0/3847037332 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400cde0 con 0x7f0bc8074c30 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.449+0000 7f0bbf400640 1 -- 172.21.15.7:0/3847037332 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb40035f0 con 0x7f0bc8074c30 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.449+0000 7f0bbf400640 1 -- 172.21.15.7:0/3847037332 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4010690 con 0x7f0bc8074c30 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.449+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc819b230 con 0x7f0bc8074c30 2024-04-20T21:34:34.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.449+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc81a74f0 con 0x7f0bc8074c30 2024-04-20T21:34:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.452+0000 7f0bd040b740 1 Processor -- start 2024-04-20T21:34:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.452+0000 7f0bd040b740 1 -- start start 2024-04-20T21:34:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e10c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6d6b150 con 0x55a6e6e05ca0 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6d11370 con 0x55a6e6e05380 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bd040b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6d67110 con 0x55a6e6e0e880 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45288/0 (socket says 172.21.15.7:45288) 2024-04-20T21:34:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 -- 172.21.15.7:0/1022790057 learned_addr learned my addr 172.21.15.7:0/1022790057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 -- 172.21.15.7:0/1022790057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 msgr2=0x55a6e6e10c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e10c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 -- 172.21.15.7:0/1022790057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 msgr2=0x55a6e6e0e340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.453+0000 7f0bc7400640 1 -- 172.21.15.7:0/1022790057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6e6b477d0 con 0x55a6e6e05380 2024-04-20T21:34:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.454+0000 7f0bc7400640 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002c7a0 tx=0x7f0bc0005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a3213e1d2e20f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.454+0000 7f0bb3e00640 1 -- 172.21.15.7:0/1022790057 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0043070 con 0x55a6e6e05380 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.454+0000 7f0bb3e00640 1 -- 172.21.15.7:0/1022790057 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc0003e50 con 0x55a6e6e05380 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.454+0000 7f0bb3e00640 1 -- 172.21.15.7:0/1022790057 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0039040 con 0x55a6e6e05380 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.454+0000 7f0bbf400640 1 -- 172.21.15.7:0/3847037332 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f0bb40037a0 con 0x7f0bc8074c30 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 -- 172.21.15.7:0/1022790057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 msgr2=0x55a6e6e05760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002c7a0 tx=0x7f0bc0005f80 comp rx=0 tx=0).stop 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 -- 172.21.15.7:0/1022790057 shutdown_connections 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e10c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 --2- 172.21.15.7:0/1022790057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6e05760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 -- 172.21.15.7:0/1022790057 >> 172.21.15.7:0/1022790057 conn(0x55a6e6dffab0 msgr2=0x55a6e6dffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 -- 172.21.15.7:0/1022790057 shutdown_connections 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.455+0000 7f0bd040b740 1 -- 172.21.15.7:0/1022790057 wait complete. 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.456+0000 7f0bd040b740 1 -- start start 2024-04-20T21:34:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.456+0000 7f0bd040b740 1 Processor -- start 2024-04-20T21:34:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e0c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6e11610 con 0x55a6e6e05380 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6d24740 con 0x55a6e6e05ca0 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a6e6e11790 con 0x55a6e6e0e880 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bd040b740 10 client.0 _ll_register_callbacks cb 0x55a6e6d1fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53000/0 (socket says 172.21.15.7:53000) 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc7400640 1 -- 172.21.15.7:0/2809300 learned_addr learned my addr 172.21.15.7:0/2809300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.457+0000 7f0bc6000640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e0c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 -- 172.21.15.7:0/2809300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 msgr2=0x55a6e6e0c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e0c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 -- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 msgr2=0x55a6e6e0be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 -- 172.21.15.7:0/2809300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6e6a7d3f0 con 0x55a6e6e05380 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bbf400640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 0x7f0bac0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bc7400640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009a60 tx=0x7f0bc0004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987b255d17b9cbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:34:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc004c070 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.458+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0bc0003b70 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc0003e50 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bbf400640 1 -- 172.21.15.7:0/3847037332 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0bb40d0c20 con 0x7f0bc8074c30 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0bc0058050 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bd040b740 10 client.15465 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6e6a3d0f0 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 0x7f0bac0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.459+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc0003e50 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f0bc0048860 con 0x55a6e6e05380 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 10 client.15465 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 20 client.15465 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e1ea90 0x55a6e6e20e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 4 client.15465 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6e6d78b70 con 0x55a6e6e1ea90 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 0x7f0bac0a63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55a6e6e0cee0 tx=0x7f0bb8005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.460+0000 7f0bc6000640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e1ea90 0x55a6e6e20e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.461+0000 7f0bc6000640 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e1ea90 0x55a6e6e20e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d185458dc56f80 server_cookie=e6dd5c4882398567 in_seq=0 out_seq=0 2024-04-20T21:34:34.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.461+0000 7f0bb0c00640 10 client.15465 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bb0c00640 1 -- 172.21.15.7:0/2809300 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6e6d78b70 con 0x55a6e6e1ea90 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bb0c00640 10 client.15465 handle_command_reply: tid=0 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 client.15465 shutdown 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 msgr2=0x55a6e6e0b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009a60 tx=0x7f0bc0004650 comp rx=0 tx=0).stop 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 shutdown_connections 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e1ea90 0x55a6e6e20e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e0e880 0x55a6e6e0c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 --2- 172.21.15.7:0/2809300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05380 0x55a6e6e0b910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 >> 172.21.15.7:0/2809300 conn(0x55a6e6dffab0 msgr2=0x55a6e6e01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bb0c00640 0 client.15465 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 shutdown_connections 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.462+0000 7f0bd040b740 1 -- 172.21.15.7:0/2809300 wait complete. 2024-04-20T21:34:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.463+0000 7f0bd040b740 20 client.15465 trim_cache size 0 max 16384 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.482+0000 7f0bd040b740 1 Processor -- start 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.482+0000 7f0bd040b740 1 -- start start 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6dfeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 0x55a6e6e014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b23c0 con 0x55a6e6e0e880 2024-04-20T21:34:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81afc80 con 0x55a6e6e05380 2024-04-20T21:34:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bd040b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81adad0 con 0x55a6e6e05ca0 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bc7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 0x55a6e6e014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46962/0 (socket says 172.21.15.7:46962) 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bc6000640 1 -- 172.21.15.7:0/3168803861 learned_addr learned my addr 172.21.15.7:0/3168803861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.483+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6dfeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 -- 172.21.15.7:0/3168803861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 msgr2=0x55a6e6dfeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6dfeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 -- 172.21.15.7:0/3168803861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 msgr2=0x55a6e6e014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 0x55a6e6e014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 -- 172.21.15.7:0/3168803861 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6e6e04b20 con 0x55a6e6e05ca0 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 0x55a6e6e014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40993a0 tx=0x7f0bb40d2ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d113818beaf17d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bb3400640 1 -- 172.21.15.7:0/3168803861 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb409c510 con 0x55a6e6e05ca0 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bb3400640 1 -- 172.21.15.7:0/3168803861 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb409d0d0 con 0x55a6e6e05ca0 2024-04-20T21:34:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.484+0000 7f0bb3400640 1 -- 172.21.15.7:0/3168803861 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb409c330 con 0x55a6e6e05ca0 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 -- 172.21.15.7:0/3168803861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 msgr2=0x55a6e6e0c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40993a0 tx=0x7f0bb40d2ff0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 -- 172.21.15.7:0/3168803861 shutdown_connections 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e0e880 0x55a6e6e014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e0c1b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3168803861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e05380 0x55a6e6dfeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 -- 172.21.15.7:0/3168803861 >> 172.21.15.7:0/3168803861 conn(0x55a6e6dffab0 msgr2=0x55a6e6e00db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 -- 172.21.15.7:0/3168803861 shutdown_connections 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.485+0000 7f0bd040b740 1 -- 172.21.15.7:0/3168803861 wait complete. 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 -- start start 2024-04-20T21:34:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 Processor -- start 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05380 0x55a6e6e21f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 0x55a6e6e83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81ae140 con 0x55a6e6e05ca0 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81af5b0 con 0x55a6e6e0e880 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.486+0000 7f0bd040b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81ae810 con 0x55a6e6e05380 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bd040b740 10 client.0 _ll_register_callbacks cb 0x7f0bb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53024/0 (socket says 172.21.15.7:53024) 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc6000640 1 -- 172.21.15.7:0/3018583787 learned_addr learned my addr 172.21.15.7:0/3018583787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 0x55a6e6e83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05380 0x55a6e6e21f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 -- 172.21.15.7:0/3018583787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05380 msgr2=0x55a6e6e21f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05380 0x55a6e6e21f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 -- 172.21.15.7:0/3018583787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 msgr2=0x55a6e6e22490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc7400640 1 -- 172.21.15.7:0/3018583787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6e6e841b0 con 0x55a6e6e0e880 2024-04-20T21:34:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.487+0000 7f0bc6000640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:34.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bc7400640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 0x55a6e6e83c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0048d00 tx=0x7f0bc00325d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91456ef085d98393 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc004c040 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0bc0003b70 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb409d0d0 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0bc0061050 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.488+0000 7f0bd040b740 10 client.25586 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a6e6e853f0 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb409d0d0 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f0bc003f650 con 0x55a6e6e0e880 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 10 client.25586 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 10 client.25586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 20 client.25586 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e85b10 0x55a6e6e87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 4 client.25586 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.489+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6e6d729e0 con 0x55a6e6e85b10 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.490+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e85b10 0x55a6e6e87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.490+0000 7f0bc6a00640 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e85b10 0x55a6e6e87f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09a09efffa0056d server_cookie=95daf38fdbef7ce2 in_seq=0 out_seq=0 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.490+0000 7f0bb0c00640 10 client.25586 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.490+0000 7f0bb0c00640 1 -- 172.21.15.7:0/3018583787 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a6e6d729e0 con 0x55a6e6e85b10 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.490+0000 7f0bb0c00640 10 client.25586 handle_command_reply: tid=0 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 client.25586 shutdown 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 msgr2=0x55a6e6e83c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 0x55a6e6e83c70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0048d00 tx=0x7f0bc00325d0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 shutdown_connections 2024-04-20T21:34:34.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a6e6e0e880 0x55a6e6e83c70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a6e6e85b10 0x55a6e6e87f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bb0c00640 0 client.25586 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a6e6e05ca0 0x55a6e6e22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 --2- 172.21.15.7:0/3018583787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a6e6e05380 0x55a6e6e21f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 >> 172.21.15.7:0/3018583787 conn(0x55a6e6dffab0 msgr2=0x55a6e6dfe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 shutdown_connections 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 1 -- 172.21.15.7:0/3018583787 wait complete. 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.491+0000 7f0bd040b740 20 client.25586 trim_cache size 0 max 16384 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:34:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:34:34.494 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:34.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 msgr2=0x7f0bac0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 0x7f0bac0a63b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55a6e6e0cee0 tx=0x7f0bb8005f90 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 msgr2=0x7f0bc81a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400ea60 tx=0x7f0bb400ef20 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 shutdown_connections 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0bac0a3ef0 0x7f0bac0a63b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bc8074c30 0x7f0bc81a0ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0bc810e620 0x7f0bc819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 --2- 172.21.15.7:0/3847037332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0bc8074310 0x7f0bc81a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 >> 172.21.15.7:0/3847037332 conn(0x7f0bc80fd8b0 msgr2=0x7f0bc80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 shutdown_connections 2024-04-20T21:34:34.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.492+0000 7f0bc7e00640 1 -- 172.21.15.7:0/3847037332 wait complete. 2024-04-20T21:34:34.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:34:34.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:34:34.616 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:34 smithi186 ceph-mon[22157]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 -- 172.21.15.7:0/2196615366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 msgr2=0x7f114410e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 --2- 172.21.15.7:0/2196615366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114410e1c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1130009900 tx=0x7f113002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 -- 172.21.15.7:0/2196615366 shutdown_connections 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 --2- 172.21.15.7:0/2196615366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114410e1c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 --2- 172.21.15.7:0/2196615366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f114410e700 0x7f1144110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 --2- 172.21.15.7:0/2196615366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1144074310 0x7f11440746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 -- 172.21.15.7:0/2196615366 >> 172.21.15.7:0/2196615366 conn(0x7f11440fd950 msgr2=0x7f11440ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 -- 172.21.15.7:0/2196615366 shutdown_connections 2024-04-20T21:34:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.683+0000 7f1143200640 1 -- 172.21.15.7:0/2196615366 wait complete. 2024-04-20T21:34:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.684+0000 7f1143200640 1 Processor -- start 2024-04-20T21:34:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.684+0000 7f1143200640 1 -- start start 2024-04-20T21:34:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114410e700 0x7f11441967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1144104730 con 0x7f114410e700 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11441045b0 con 0x7f1144074c30 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1143200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f11441048b0 con 0x7f1144074310 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1141e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1141e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46986/0 (socket says 172.21.15.7:46986) 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.685+0000 7f1141e00640 1 -- 172.21.15.7:0/1833479004 learned_addr learned my addr 172.21.15.7:0/1833479004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141400640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1142800640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114410e700 0x7f11441967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 -- 172.21.15.7:0/1833479004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 msgr2=0x7f114419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 -- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114410e700 msgr2=0x7f11441967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114410e700 0x7f11441967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 -- 172.21.15.7:0/1833479004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1144196fd0 con 0x7f1144074310 2024-04-20T21:34:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.686+0000 7f1141e00640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9f0 tx=0x7f113400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.687+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113400cd80 con 0x7f1144074310 2024-04-20T21:34:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.687+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1134010910 con 0x7f1144074310 2024-04-20T21:34:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.687+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134018980 con 0x7f1144074310 2024-04-20T21:34:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.687+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11441972c0 con 0x7f1144074310 2024-04-20T21:34:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.687+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11441a3050 con 0x7f1144074310 2024-04-20T21:34:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.688+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114404fd60 con 0x7f1144074310 2024-04-20T21:34:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.690+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f1134010ac0 con 0x7f1144074310 2024-04-20T21:34:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.692+0000 7f113aa00640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 0x7f11280a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.692+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f11340d0dd0 con 0x7f1144074310 2024-04-20T21:34:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.692+0000 7f1141400640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 0x7f11280a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.693+0000 7f1141400640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 0x7f11280a6400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1130005ab0 tx=0x7f1130005a20 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:34 smithi016 ceph-mon[21409]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.696+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f113409d280 con 0x7f1144074310 2024-04-20T21:34:34.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:34 smithi007 ceph-mon[16867]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:34:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.996+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f114402d5d0 con 0x7f1144074310 2024-04-20T21:34:34.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.997+0000 7f113aa00640 1 -- 172.21.15.7:0/1833479004 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11869 (secure 0 0 0) 0x7f113409cb80 con 0x7f1144074310 2024-04-20T21:34:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:34:34.996 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:33:20.441301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:34:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 msgr2=0x7f11280a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 0x7f11280a6400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1130005ab0 tx=0x7f1130005a20 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 msgr2=0x7f114419b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f113400e9f0 tx=0x7f113400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 shutdown_connections 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f11280a3f40 0x7f11280a6400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1144074c30 0x7f114419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f114410e700 0x7f11441967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 --2- 172.21.15.7:0/1833479004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1144074310 0x7f114419b130 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:34.999+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 >> 172.21.15.7:0/1833479004 conn(0x7f11440fd950 msgr2=0x7f11440fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.000+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 shutdown_connections 2024-04-20T21:34:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.000+0000 7f1143200640 1 -- 172.21.15.7:0/1833479004 wait complete. 2024-04-20T21:34:35.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 -- 172.21.15.7:0/2858086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c104b90 msgr2=0x7fdc7c108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 --2- 172.21.15.7:0/2858086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c104b90 0x7fdc7c108bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009900 tx=0x7fdc6802bc30 comp rx=0 tx=0).stop 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 -- 172.21.15.7:0/2858086667 shutdown_connections 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 --2- 172.21.15.7:0/2858086667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c104b90 0x7fdc7c108bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 --2- 172.21.15.7:0/2858086667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1092b0 0x7fdc7c110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 --2- 172.21.15.7:0/2858086667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 -- 172.21.15.7:0/2858086667 >> 172.21.15.7:0/2858086667 conn(0x7fdc7c0ffa40 msgr2=0x7fdc7c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 -- 172.21.15.7:0/2858086667 shutdown_connections 2024-04-20T21:34:35.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.186+0000 7fdc83000640 1 -- 172.21.15.7:0/2858086667 wait complete. 2024-04-20T21:34:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.187+0000 7fdc83000640 1 Processor -- start 2024-04-20T21:34:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.187+0000 7fdc83000640 1 -- start start 2024-04-20T21:34:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1092b0 0x7fdc7c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c19a6d0 0x7fdc7c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c114390 con 0x7fdc7c1092b0 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c114210 con 0x7fdc7c19a6d0 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc83000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c114510 con 0x7fdc7c1041e0 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc81200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1092b0 0x7fdc7c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47008/0 (socket says 172.21.15.7:47008) 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.188+0000 7fdc81c00640 1 -- 172.21.15.7:0/3154519443 learned_addr learned my addr 172.21.15.7:0/3154519443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc82600640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c19a6d0 0x7fdc7c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 -- 172.21.15.7:0/3154519443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c19a6d0 msgr2=0x7fdc7c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c19a6d0 0x7fdc7c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 -- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1092b0 msgr2=0x7fdc7c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1092b0 0x7fdc7c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 -- 172.21.15.7:0/3154519443 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c19b2d0 con 0x7fdc7c1041e0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.189+0000 7fdc81c00640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e9f0 tx=0x7fdc7000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.190+0000 7fdc7a000640 1 -- 172.21.15.7:0/3154519443 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000cd80 con 0x7fdc7c1041e0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.190+0000 7fdc7a000640 1 -- 172.21.15.7:0/3154519443 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc70010910 con 0x7fdc7c1041e0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.190+0000 7fdc7a000640 1 -- 172.21.15.7:0/3154519443 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70018990 con 0x7fdc7c1041e0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.190+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c1a73b0 con 0x7fdc7c1041e0 2024-04-20T21:34:35.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.190+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c1a77b0 con 0x7fdc7c1041e0 2024-04-20T21:34:35.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.194+0000 7fdc8743a740 1 Processor -- start 2024-04-20T21:34:35.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.194+0000 7fdc8743a740 1 -- start start 2024-04-20T21:34:35.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 0x55f7c5d46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5ca0050 con 0x55f7c5d44670 2024-04-20T21:34:35.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5c46490 con 0x55f7c5d3ba90 2024-04-20T21:34:35.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc8743a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5c9c010 con 0x55f7c5d3b170 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc7a000640 1 -- 172.21.15.7:0/3154519443 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7fdc70010ac0 con 0x7fdc7c1041e0 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc82600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc81200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 0x55f7c5d46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc82600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47016/0 (socket says 172.21.15.7:47016) 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc82600640 1 -- 172.21.15.7:0/1297942459 learned_addr learned my addr 172.21.15.7:0/1297942459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.195+0000 7fdc81c00640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81200640 1 -- 172.21.15.7:0/1297942459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 msgr2=0x55f7c5d3b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81200640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81200640 1 -- 172.21.15.7:0/1297942459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 msgr2=0x55f7c5d44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81200640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81200640 1 -- 172.21.15.7:0/1297942459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c5c4e070 con 0x55f7c5d44670 2024-04-20T21:34:35.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc82600640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.196+0000 7fdc81c00640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc81200640 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 0x55f7c5d46a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036040 tx=0x7fdc680095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e1fc17b55436a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc7a000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 0x7fdc640a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc6ea00640 1 -- 172.21.15.7:0/1297942459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68045070 con 0x55f7c5d44670 2024-04-20T21:34:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc7a000640 1 -- 172.21.15.7:0/3154519443 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdc700d0d10 con 0x7fdc7c1041e0 2024-04-20T21:34:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc6ea00640 1 -- 172.21.15.7:0/1297942459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6802c880 con 0x55f7c5d44670 2024-04-20T21:34:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc6ea00640 1 -- 172.21.15.7:0/1297942459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004b50 con 0x55f7c5d44670 2024-04-20T21:34:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.197+0000 7fdc82600640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 0x7fdc640a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.198+0000 7fdc82600640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 0x7fdc640a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc74005f40 tx=0x7fdc74009600 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.201+0000 7fdc8743a740 1 -- 172.21.15.7:0/1297942459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 msgr2=0x55f7c5d46a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.201+0000 7fdc8743a740 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 0x55f7c5d46a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036040 tx=0x7fdc680095e0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 -- 172.21.15.7:0/1297942459 shutdown_connections 2024-04-20T21:34:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d44670 0x55f7c5d46a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 --2- 172.21.15.7:0/1297942459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d3b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 -- 172.21.15.7:0/1297942459 >> 172.21.15.7:0/1297942459 conn(0x55f7c5d358a0 msgr2=0x55f7c5d35c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 -- 172.21.15.7:0/1297942459 shutdown_connections 2024-04-20T21:34:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 -- 172.21.15.7:0/1297942459 wait complete. 2024-04-20T21:34:35.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 -- start start 2024-04-20T21:34:35.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.202+0000 7fdc8743a740 1 Processor -- start 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d41bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d44670 0x55f7c5d42110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5d47400 con 0x55f7c5d3ba90 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5c59860 con 0x55f7c5d44670 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f7c5d47580 con 0x55f7c5d3b170 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc8743a740 10 client.0 _ll_register_callbacks cb 0x55f7c5c27fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.203+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47032/0 (socket says 172.21.15.7:47032) 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 -- 172.21.15.7:0/836824496 learned_addr learned my addr 172.21.15.7:0/836824496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81200640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d41bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc82600640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d44670 0x55f7c5d42110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 -- 172.21.15.7:0/836824496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d44670 msgr2=0x55f7c5d42110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d44670 0x55f7c5d42110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 -- 172.21.15.7:0/836824496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 msgr2=0x55f7c5d41bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d41bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 -- 172.21.15.7:0/836824496 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c5c5d950 con 0x55f7c5d3b170 2024-04-20T21:34:35.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc81c00640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc700ce360 tx=0x7fdc7009c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40b4d9b53fdc752 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.204+0000 7fdc63400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70018d30 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdc7009bfc0 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc6802c880 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc7009cb70 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc8743a740 10 client.25600 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f7c5c2b820 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.205+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6802c880 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fdc700d5040 con 0x55f7c5d3b170 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 10 client.25600 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 10 client.25600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 20 client.25600 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5d5cd90 0x55f7c5d5f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 4 client.25600 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7c5cada70 con 0x55f7c5d5cd90 2024-04-20T21:34:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.206+0000 7fdc81200640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5d5cd90 0x55f7c5d5f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.207+0000 7fdc81200640 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5d5cd90 0x55f7c5d5f1c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=875ec627a0cdd451 server_cookie=7cf2fa73f46789f1 in_seq=0 out_seq=0 2024-04-20T21:34:35.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.207+0000 7fdc63400640 10 client.25600 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.207+0000 7fdc63400640 1 -- 172.21.15.7:0/836824496 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7c5cada70 con 0x55f7c5d5cd90 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.207+0000 7fdc63400640 10 client.25600 handle_command_reply: tid=0 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 client.25600 shutdown 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 msgr2=0x55f7c5d41690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc700ce360 tx=0x7fdc7009c940 comp rx=0 tx=0).stop 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 shutdown_connections 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5d5cd90 0x55f7c5d5f1c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d44670 0x55f7c5d42110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d41bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 --2- 172.21.15.7:0/836824496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d3b170 0x55f7c5d41690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc63400640 0 client.25600 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 >> 172.21.15.7:0/836824496 conn(0x55f7c5d358a0 msgr2=0x55f7c5d375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 shutdown_connections 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 1 -- 172.21.15.7:0/836824496 wait complete. 2024-04-20T21:34:35.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.208+0000 7fdc8743a740 20 client.25600 trim_cache size 0 max 16384 2024-04-20T21:34:35.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 Processor -- start 2024-04-20T21:34:35.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 -- start start 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1ae0b0 con 0x55f7c5d3b170 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1ab970 con 0x55f7c5d3ba90 2024-04-20T21:34:35.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc8743a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1a97e0 con 0x55f7c5d44670 2024-04-20T21:34:35.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.223+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53094/0 (socket says 172.21.15.7:53094) 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47048/0 (socket says 172.21.15.7:47048) 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc81c00640 1 -- 172.21.15.7:0/4009816640 learned_addr learned my addr 172.21.15.7:0/4009816640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc81200640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 -- 172.21.15.7:0/4009816640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 msgr2=0x55f7c5d58b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d58b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 -- 172.21.15.7:0/4009816640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 msgr2=0x55f7c5d43dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc82600640 1 -- 172.21.15.7:0/4009816640 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7c59b23f0 con 0x55f7c5d44670 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc81200640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d58b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.224+0000 7fdc81c00640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.225+0000 7fdc82600640 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009b80 tx=0x7fdc74016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71cb741d813c65ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.225+0000 7fdc6e000640 1 -- 172.21.15.7:0/4009816640 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74004b20 con 0x55f7c5d44670 2024-04-20T21:34:35.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.225+0000 7fdc6e000640 1 -- 172.21.15.7:0/4009816640 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74011d20 con 0x55f7c5d44670 2024-04-20T21:34:35.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.225+0000 7fdc6e000640 1 -- 172.21.15.7:0/4009816640 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740046a0 con 0x55f7c5d44670 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- 172.21.15.7:0/4009816640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 msgr2=0x55f7c5d347e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009b80 tx=0x7fdc74016000 comp rx=0 tx=0).stop 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- 172.21.15.7:0/4009816640 shutdown_connections 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d58b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d347e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 --2- 172.21.15.7:0/4009816640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3b170 0x55f7c5d43dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- 172.21.15.7:0/4009816640 >> 172.21.15.7:0/4009816640 conn(0x55f7c5d358a0 msgr2=0x55f7c5d340d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- 172.21.15.7:0/4009816640 shutdown_connections 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- 172.21.15.7:0/4009816640 wait complete. 2024-04-20T21:34:35.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.226+0000 7fdc8743a740 1 -- start start 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 Processor -- start 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d3e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1a9e30 con 0x55f7c5d3ba90 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1ab2a0 con 0x55f7c5d3b170 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c1aa500 con 0x55f7c5d44670 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.227+0000 7fdc8743a740 10 client.0 _ll_register_callbacks cb 0x7fdc70009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45392/0 (socket says 172.21.15.7:45392) 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 -- 172.21.15.7:0/2550747308 learned_addr learned my addr 172.21.15.7:0/2550747308 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc82600640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d3e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81200640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 -- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 msgr2=0x55f7c5d3e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d3e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 -- 172.21.15.7:0/2550747308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 msgr2=0x55f7c5d3e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d3e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 -- 172.21.15.7:0/2550747308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7c5a7c7d0 con 0x55f7c5d3b170 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81200640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d3e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc81c00640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc700d4560 tx=0x7fdc70098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe9c154f7811604 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc63400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.228+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700988c0 con 0x55f7c5d3b170 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc74011d20 con 0x55f7c5d3b170 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdc7009bfc0 con 0x55f7c5d3b170 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdc700a0e70 con 0x55f7c5d3b170 2024-04-20T21:34:35.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 10 client.25610 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f7c5dc32a0 con 0x55f7c5d3b170 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc74011d20 con 0x55f7c5d3b170 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7fdc700ce710 con 0x55f7c5d3b170 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 10 client.25610 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 10 client.25610 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 20 client.25610 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5dc3a10 0x55f7c5dc5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 4 client.25610 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:35.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.229+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7c5ca78e0 con 0x55f7c5dc3a10 2024-04-20T21:34:35.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.230+0000 7fdc81200640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5dc3a10 0x55f7c5dc5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.230+0000 7fdc81200640 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5dc3a10 0x55f7c5dc5e00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad716abeac6b15aa server_cookie=35cf46809f0232a in_seq=0 out_seq=0 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.230+0000 7fdc63400640 10 client.25610 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.230+0000 7fdc63400640 1 -- 172.21.15.7:0/2550747308 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7c5ca78e0 con 0x55f7c5dc3a10 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.230+0000 7fdc63400640 10 client.25610 handle_command_reply: tid=0 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 client.25610 shutdown 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 msgr2=0x55f7c5d555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc700d4560 tx=0x7fdc70098cd0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 shutdown_connections 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f7c5d44670 0x55f7c5d3e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f7c5d3ba90 0x55f7c5d3e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55f7c5dc3a10 0x55f7c5dc5e00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 --2- 172.21.15.7:0/2550747308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f7c5d3b170 0x55f7c5d555a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc63400640 0 client.25610 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:34:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 >> 172.21.15.7:0/2550747308 conn(0x55f7c5d358a0 msgr2=0x55f7c5d374d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 shutdown_connections 2024-04-20T21:34:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 1 -- 172.21.15.7:0/2550747308 wait complete. 2024-04-20T21:34:35.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.231+0000 7fdc8743a740 20 client.25610 trim_cache size 0 max 16384 2024-04-20T21:34:35.234 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 msgr2=0x7fdc640a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 0x7fdc640a6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdc74005f40 tx=0x7fdc74009600 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 msgr2=0x7fdc7c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000e9f0 tx=0x7fdc7000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 shutdown_connections 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdc640a3f50 0x7fdc640a6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc7c19a6d0 0x7fdc7c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc7c1092b0 0x7fdc7c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 --2- 172.21.15.7:0/3154519443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc7c1041e0 0x7fdc7c1a0610 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 >> 172.21.15.7:0/3154519443 conn(0x7fdc7c0ffa40 msgr2=0x7fdc7c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.232+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 shutdown_connections 2024-04-20T21:34:35.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.233+0000 7fdc83000640 1 -- 172.21.15.7:0/3154519443 wait complete. 2024-04-20T21:34:35.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 -- 172.21.15.7:0/3379159810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5810c510 msgr2=0x7f0a5810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 --2- 172.21.15.7:0/3379159810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5810c510 0x7f0a5810e9b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009900 tx=0x7f0a5002bcd0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 -- 172.21.15.7:0/3379159810 shutdown_connections 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 --2- 172.21.15.7:0/3379159810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a5810c510 0x7f0a5810e9b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 --2- 172.21.15.7:0/3379159810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a58103af0 0x7f0a5810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 --2- 172.21.15.7:0/3379159810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 -- 172.21.15.7:0/3379159810 >> 172.21.15.7:0/3379159810 conn(0x7f0a58078010 msgr2=0x7f0a58078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 -- 172.21.15.7:0/3379159810 shutdown_connections 2024-04-20T21:34:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.436+0000 7f0a60400640 1 -- 172.21.15.7:0/3379159810 wait complete. 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.437+0000 7f0a60400640 1 Processor -- start 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 -- start start 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a5810c510 0x7f0a5819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a58112340 con 0x7f0a58103af0 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581121c0 con 0x7f0a581031d0 2024-04-20T21:34:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a60400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581124c0 con 0x7f0a5810c510 2024-04-20T21:34:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53132/0 (socket says 172.21.15.7:53132) 2024-04-20T21:34:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a5d000640 1 -- 172.21.15.7:0/1967725091 learned_addr learned my addr 172.21.15.7:0/1967725091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a5da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.438+0000 7f0a5e400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a5810c510 0x7f0a5819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 -- 172.21.15.7:0/1967725091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a5810c510 msgr2=0x7f0a5819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a5810c510 0x7f0a5819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 -- 172.21.15.7:0/1967725091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 msgr2=0x7f0a581a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 -- 172.21.15.7:0/1967725091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5819afe0 con 0x7f0a58103af0 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5da00640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.439+0000 7f0a5d000640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e9f0 tx=0x7f0a4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.440+0000 7f0a56000640 1 -- 172.21.15.7:0/1967725091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400cd80 con 0x7f0a58103af0 2024-04-20T21:34:35.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.440+0000 7f0a56000640 1 -- 172.21.15.7:0/1967725091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44005500 con 0x7f0a58103af0 2024-04-20T21:34:35.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.440+0000 7f0a56000640 1 -- 172.21.15.7:0/1967725091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44010690 con 0x7f0a58103af0 2024-04-20T21:34:35.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.440+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5819b2d0 con 0x7f0a58103af0 2024-04-20T21:34:35.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.440+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a581a74f0 con 0x7f0a58103af0 2024-04-20T21:34:35.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.443+0000 7f0a63183740 1 Processor -- start 2024-04-20T21:34:35.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.443+0000 7f0a63183740 1 -- start start 2024-04-20T21:34:35.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e96fb0 0x55b135e97390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135ea04b0 0x55b135ea28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b135dfc050 con 0x55b135e96fb0 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b135da2490 con 0x55b135e978d0 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a63183740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b135df8010 con 0x55b135ea04b0 2024-04-20T21:34:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a56000640 1 -- 172.21.15.7:0/1967725091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f0a440046c0 con 0x7f0a58103af0 2024-04-20T21:34:35.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a5e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e96fb0 0x55b135e97390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a5da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a5da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45428/0 (socket says 172.21.15.7:45428) 2024-04-20T21:34:35.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.444+0000 7f0a5da00640 1 -- 172.21.15.7:0/3112388494 learned_addr learned my addr 172.21.15.7:0/3112388494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5d000640 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135ea04b0 0x55b135ea28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5da00640 1 -- 172.21.15.7:0/3112388494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135ea04b0 msgr2=0x55b135ea28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5da00640 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135ea04b0 0x55b135ea28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5da00640 1 -- 172.21.15.7:0/3112388494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e96fb0 msgr2=0x55b135e97390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5da00640 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e96fb0 0x55b135e97390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.445+0000 7f0a5da00640 1 -- 172.21.15.7:0/3112388494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b135daa070 con 0x55b135e978d0 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.446+0000 7f0a5da00640 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011860 tx=0x7f0a4c011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c80e352d1b8c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.446+0000 7f0a4aa00640 1 -- 172.21.15.7:0/3112388494 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0112f0 con 0x55b135e978d0 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.446+0000 7f0a4aa00640 1 -- 172.21.15.7:0/3112388494 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c003e50 con 0x55b135e978d0 2024-04-20T21:34:35.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.447+0000 7f0a56000640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 0x7f0a400a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.447+0000 7f0a56000640 1 -- 172.21.15.7:0/1967725091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0a440d0d70 con 0x7f0a58103af0 2024-04-20T21:34:35.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.447+0000 7f0a63183740 1 -- 172.21.15.7:0/3112388494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 msgr2=0x55b135e9ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.447+0000 7f0a63183740 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011860 tx=0x7f0a4c011d20 comp rx=0 tx=0).stop 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.447+0000 7f0a5e400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 0x7f0a400a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 -- 172.21.15.7:0/3112388494 shutdown_connections 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9ff70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135ea04b0 0x55b135ea28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 --2- 172.21.15.7:0/3112388494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e96fb0 0x55b135e97390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 -- 172.21.15.7:0/3112388494 >> 172.21.15.7:0/3112388494 conn(0x55b135e916e0 msgr2=0x55b135e91ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 -- 172.21.15.7:0/3112388494 shutdown_connections 2024-04-20T21:34:35.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a5e400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 0x7f0a400a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005ab0 tx=0x7f0a5002e040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.448+0000 7f0a63183740 1 -- 172.21.15.7:0/3112388494 wait complete. 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 -- start start 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 Processor -- start 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135e9de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b135ea3340 con 0x55b135ea04b0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b135db5860 con 0x55b135e978d0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55b135ea34c0 con 0x55b135e96fb0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.449+0000 7f0a63183740 10 client.0 _ll_register_callbacks cb 0x55b135d83fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135e9de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45432/0 (socket says 172.21.15.7:45432) 2024-04-20T21:34:35.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135e9de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53158/0 (socket says 172.21.15.7:53158) 2024-04-20T21:34:35.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5e400640 1 -- 172.21.15.7:0/1632391800 learned_addr learned my addr 172.21.15.7:0/1632391800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5da00640 1 -- 172.21.15.7:0/1632391800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 msgr2=0x55b135e9d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5da00640 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5da00640 1 -- 172.21.15.7:0/1632391800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 msgr2=0x55b135e9de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.450+0000 7f0a5da00640 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135e9de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a5da00640 1 -- 172.21.15.7:0/1632391800 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b135db9950 con 0x55b135e96fb0 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a5da00640 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9d3a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00f790 tx=0x7f0a4c000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85cfd23406dd599 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:35.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00f9a0 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0a4c00fb40 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.451+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4c003e50 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c032050 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 10 client.25618 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b135d87820 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c01f040 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f0a4c018070 con 0x55b135e96fb0 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 10 client.25618 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 10 client.25618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 20 client.25618 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135eb0430 0x55b135eb2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 4 client.25618 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.452+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a4c003e50 con 0x55b135eb0430 2024-04-20T21:34:35.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.453+0000 7f0a5d000640 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135eb0430 0x55b135eb2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.453+0000 7f0a5d000640 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135eb0430 0x55b135eb2820 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cf8f92347779b52 server_cookie=52143918134575 in_seq=0 out_seq=0 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.453+0000 7f0a3f400640 10 client.25618 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a3f400640 1 -- 172.21.15.7:0/1632391800 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0a4c003e50 con 0x55b135eb0430 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a3f400640 10 client.25618 handle_command_reply: tid=0 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a63183740 1 client.25618 shutdown 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 msgr2=0x55b135e9d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9d3a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00f790 tx=0x7f0a4c000f40 comp rx=0 tx=0).stop 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 shutdown_connections 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.454+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135eb0430 0x55b135eb2820 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e978d0 0x55b135e9d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135e9de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 --2- 172.21.15.7:0/1632391800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9d3a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 >> 172.21.15.7:0/1632391800 conn(0x55b135e916e0 msgr2=0x55b135e93430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a3f400640 0 client.25618 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 shutdown_connections 2024-04-20T21:34:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 1 -- 172.21.15.7:0/1632391800 wait complete. 2024-04-20T21:34:35.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.455+0000 7f0a63183740 20 client.25618 trim_cache size 0 max 16384 2024-04-20T21:34:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.469+0000 7f0a63183740 1 Processor -- start 2024-04-20T21:34:35.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.469+0000 7f0a63183740 1 -- start start 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135ea04b0 0x55b135e8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581b2620 con 0x55b135e978d0 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581afee0 con 0x55b135ea04b0 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a63183740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581add30 con 0x55b135e96fb0 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53160/0 (socket says 172.21.15.7:53160) 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a5da00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a5d000640 1 -- 172.21.15.7:0/926589855 learned_addr learned my addr 172.21.15.7:0/926589855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.470+0000 7f0a5e400640 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135ea04b0 0x55b135e8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 -- 172.21.15.7:0/926589855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135ea04b0 msgr2=0x55b135e8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135ea04b0 0x55b135e8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 -- 172.21.15.7:0/926589855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 msgr2=0x55b135eac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 -- 172.21.15.7:0/926589855 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b135e90d60 con 0x55b135e96fb0 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5d000640 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a5da00640 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9fad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011860 tx=0x7f0a4c01b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80e0d8404fdd1da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a4a000640 1 -- 172.21.15.7:0/926589855 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c03c070 con 0x55b135e96fb0 2024-04-20T21:34:35.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a4a000640 1 -- 172.21.15.7:0/926589855 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c005980 con 0x55b135e96fb0 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.471+0000 7f0a4a000640 1 -- 172.21.15.7:0/926589855 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c004990 con 0x55b135e96fb0 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 -- 172.21.15.7:0/926589855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 msgr2=0x55b135e9fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9fad0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c011860 tx=0x7f0a4c01b040 comp rx=0 tx=0).stop 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 -- 172.21.15.7:0/926589855 shutdown_connections 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135ea04b0 0x55b135e8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135e978d0 0x55b135eac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 --2- 172.21.15.7:0/926589855 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e96fb0 0x55b135e9fad0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 -- 172.21.15.7:0/926589855 >> 172.21.15.7:0/926589855 conn(0x55b135e916e0 msgr2=0x55b135e9c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 -- 172.21.15.7:0/926589855 shutdown_connections 2024-04-20T21:34:35.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.472+0000 7f0a63183740 1 -- 172.21.15.7:0/926589855 wait complete. 2024-04-20T21:34:35.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 -- start start 2024-04-20T21:34:35.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 Processor -- start 2024-04-20T21:34:35.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 0x55b135ea4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e978d0 0x55b135ea4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581ae3a0 con 0x55b135ea04b0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581af810 con 0x55b135e96fb0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581aea70 con 0x55b135e978d0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.473+0000 7f0a63183740 10 client.0 _ll_register_callbacks cb 0x7f0a4c0116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53166/0 (socket says 172.21.15.7:53166) 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5e400640 1 -- 172.21.15.7:0/2377382376 learned_addr learned my addr 172.21.15.7:0/2377382376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 0x55b135ea4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5d000640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e978d0 0x55b135ea4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 -- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e978d0 msgr2=0x55b135ea4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e978d0 0x55b135ea4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 -- 172.21.15.7:0/2377382376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 msgr2=0x55b135ea4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5da00640 1 -- 172.21.15.7:0/2377382376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b135b0e3f0 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.474+0000 7f0a5e400640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a5da00640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 0x55b135ea4150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0117b0 tx=0x7f0a4c005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c7a8c41715cbe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c03c040 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a4c005980 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0a4c00fcf0 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0a4c00f940 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a63183740 10 client.25628 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.475+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b135bd87d0 con 0x55b135e96fb0 2024-04-20T21:34:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4c005980 con 0x55b135e96fb0 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f0a4c03f030 con 0x55b135e96fb0 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a63183740 10 client.25628 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a63183740 10 client.25628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a63183740 20 client.25628 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.476+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135f16d30 0x55b135f19120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.477+0000 7f0a63183740 4 client.25628 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.477+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b135e09a70 con 0x55b135f16d30 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.477+0000 7f0a5d000640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135f16d30 0x55b135f19120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.477+0000 7f0a5d000640 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135f16d30 0x55b135f19120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4501c6efdb8c16a server_cookie=b0824bab326522ac in_seq=0 out_seq=0 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.477+0000 7f0a3f400640 10 client.25628 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a3f400640 1 -- 172.21.15.7:0/2377382376 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b135e09a70 con 0x55b135f16d30 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a3f400640 10 client.25628 handle_command_reply: tid=0 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 client.25628 shutdown 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 msgr2=0x55b135ea4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 0x55b135ea4150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0117b0 tx=0x7f0a4c005cf0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 shutdown_connections 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55b135f16d30 0x55b135f19120 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b135ea04b0 0x55b135ea4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b135e978d0 0x55b135ea4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 --2- 172.21.15.7:0/2377382376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b135e96fb0 0x55b135ea4150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 >> 172.21.15.7:0/2377382376 conn(0x55b135e916e0 msgr2=0x55b135e9d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a3f400640 0 client.25628 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 shutdown_connections 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.478+0000 7f0a63183740 1 -- 172.21.15.7:0/2377382376 wait complete. 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a63183740 20 client.25628 trim_cache size 0 max 16384 2024-04-20T21:34:35.480 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 msgr2=0x7f0a400a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 0x7f0a400a6490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005ab0 tx=0x7f0a5002e040 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 msgr2=0x7f0a581a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e9f0 tx=0x7f0a4400edf0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 shutdown_connections 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a5810c510 0x7f0a5819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a58103af0 0x7f0a581a0be0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f0a400a3fd0 0x7f0a400a6490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 --2- 172.21.15.7:0/1967725091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a581031d0 0x7f0a581a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.479+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 >> 172.21.15.7:0/1967725091 conn(0x7f0a58078010 msgr2=0x7f0a58102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.480+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 shutdown_connections 2024-04-20T21:34:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.480+0000 7f0a60400640 1 -- 172.21.15.7:0/1967725091 wait complete. 2024-04-20T21:34:35.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:34:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 -- 172.21.15.7:0/3917310897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de0102980 msgr2=0x7f2de01069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 --2- 172.21.15.7:0/3917310897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de0102980 0x7f2de01069c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009900 tx=0x7f2dcc02bc40 comp rx=0 tx=0).stop 2024-04-20T21:34:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 -- 172.21.15.7:0/3917310897 shutdown_connections 2024-04-20T21:34:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 --2- 172.21.15.7:0/3917310897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de01070a0 0x7f2de010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 --2- 172.21.15.7:0/3917310897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de0102980 0x7f2de01069c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 --2- 172.21.15.7:0/3917310897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0101fd0 0x7f2de01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 -- 172.21.15.7:0/3917310897 >> 172.21.15.7:0/3917310897 conn(0x7f2de00fd8b0 msgr2=0x7f2de00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 -- 172.21.15.7:0/3917310897 shutdown_connections 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.670+0000 7f2de5e00640 1 -- 172.21.15.7:0/3917310897 wait complete. 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.672+0000 7f2de5e00640 1 Processor -- start 2024-04-20T21:34:35.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.672+0000 7f2de5e00640 1 -- start start 2024-04-20T21:34:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de01070a0 0x7f2de01a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 0x7f2de019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0112160 con 0x7f2de019a760 2024-04-20T21:34:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0111fe0 con 0x7f2de0101fd0 2024-04-20T21:34:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01122e0 con 0x7f2de01070a0 2024-04-20T21:34:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45450/0 (socket says 172.21.15.7:45450) 2024-04-20T21:34:35.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 -- 172.21.15.7:0/379164804 learned_addr learned my addr 172.21.15.7:0/379164804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de5400640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 0x7f2de019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de01070a0 0x7f2de01a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 -- 172.21.15.7:0/379164804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de01070a0 msgr2=0x7f2de01a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de01070a0 0x7f2de01a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 -- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 msgr2=0x7f2de019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 0x7f2de019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.673+0000 7f2de4a00640 1 -- 172.21.15.7:0/379164804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de019b360 con 0x7f2de0101fd0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2de5400640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 0x7f2de019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2de4a00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9f0 tx=0x7f2dd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2ddd600640 1 -- 172.21.15.7:0/379164804 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd400cd80 con 0x7f2de0101fd0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2ddd600640 1 -- 172.21.15.7:0/379164804 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd4010910 con 0x7f2de0101fd0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de01a73a0 con 0x7f2de0101fd0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2ddd600640 1 -- 172.21.15.7:0/379164804 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4018990 con 0x7f2de0101fd0 2024-04-20T21:34:35.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.674+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de01a77a0 con 0x7f2de0101fd0 2024-04-20T21:34:35.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.678+0000 7f2ddd600640 1 -- 172.21.15.7:0/379164804 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f2dd4010ac0 con 0x7f2de0101fd0 2024-04-20T21:34:35.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 Processor -- start 2024-04-20T21:34:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 -- start start 2024-04-20T21:34:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2ddd600640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 0x7f2dc80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2ddd600640 1 -- 172.21.15.7:0/379164804 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2dd40d0dc0 con 0x7f2de0101fd0 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858a42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a44a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56185899e050 con 0x561858a39170 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561858944490 con 0x561858a42670 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.679+0000 7f2dea053740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56185899a010 con 0x561858a39a90 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.680+0000 7f2de5400640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 0x7f2dc80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.680+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.680+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53186/0 (socket says 172.21.15.7:53186) 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.680+0000 7f2de4a00640 1 -- 172.21.15.7:0/1100211962 learned_addr learned my addr 172.21.15.7:0/1100211962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.680+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858a42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 -- 172.21.15.7:0/1100211962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 msgr2=0x561858a42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858a42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 -- 172.21.15.7:0/1100211962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 msgr2=0x561858a44a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:34:35.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a44a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 -- 172.21.15.7:0/1100211962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185894c070 con 0x561858a39170 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de5400640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 0x7f2dc80a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005fd0 tx=0x7f2dd8009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c6a0 tx=0x7f2dd409ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d8d482e56de5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2dd1c00640 1 -- 172.21.15.7:0/1100211962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40a57e0 con 0x561858a39170 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2dd1c00640 1 -- 172.21.15.7:0/1100211962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd40bd070 con 0x561858a39170 2024-04-20T21:34:35.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.681+0000 7f2dd1c00640 1 -- 172.21.15.7:0/1100211962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd409c1c0 con 0x561858a39170 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.682+0000 7f2dea053740 1 -- 172.21.15.7:0/1100211962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 msgr2=0x561858a39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.682+0000 7f2dea053740 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400c6a0 tx=0x7f2dd409ca70 comp rx=0 tx=0).stop 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 -- 172.21.15.7:0/1100211962 shutdown_connections 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a44a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858a42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 --2- 172.21.15.7:0/1100211962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858a39550 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 -- 172.21.15.7:0/1100211962 >> 172.21.15.7:0/1100211962 conn(0x561858a338a0 msgr2=0x561858a33c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 -- 172.21.15.7:0/1100211962 shutdown_connections 2024-04-20T21:34:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 -- 172.21.15.7:0/1100211962 wait complete. 2024-04-20T21:34:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.683+0000 7f2dea053740 1 -- start start 2024-04-20T21:34:35.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 Processor -- start 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a401d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561858a45460 con 0x561858a39a90 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561858957860 con 0x561858a39170 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.684+0000 7f2dea053740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561858a455e0 con 0x561858a42670 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2dea053740 10 client.0 _ll_register_callbacks cb 0x561858925fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45464/0 (socket says 172.21.15.7:45464) 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 -- 172.21.15.7:0/1789265864 learned_addr learned my addr 172.21.15.7:0/1789265864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de5400640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 -- 172.21.15.7:0/1789265864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 msgr2=0x561858a401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 -- 172.21.15.7:0/1789265864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 msgr2=0x561858a3fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.685+0000 7f2de4a00640 1 -- 172.21.15.7:0/1789265864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185895b950 con 0x561858a39170 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2de4a00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dd409cb00 tx=0x7f2dd40a1070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ca2c9d2d48a61d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd409bfc0 con 0x561858a39170 2024-04-20T21:34:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2dd409c160 con 0x561858a39170 2024-04-20T21:34:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd40bd070 con 0x561858a39170 2024-04-20T21:34:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2dd40aa050 con 0x561858a39170 2024-04-20T21:34:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dea053740 10 client.25637 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.686+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561858929820 con 0x561858a39170 2024-04-20T21:34:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd40bd070 con 0x561858a39170 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f2dd40b7070 con 0x561858a39170 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 10 client.25637 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 10 client.25637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 20 client.25637 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858a525b0 0x561858a549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 4 client.25637 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.687+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618589aba70 con 0x561858a525b0 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.688+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858a525b0 0x561858a549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.688+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858a525b0 0x561858a549a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a949f959ca1fb68 server_cookie=41eb5218f909d071 in_seq=0 out_seq=0 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.688+0000 7f2dc6000640 10 client.25637 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dc6000640 1 -- 172.21.15.7:0/1789265864 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618589aba70 con 0x561858a525b0 2024-04-20T21:34:35.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dc6000640 10 client.25637 handle_command_reply: tid=0 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 client.25637 shutdown 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 msgr2=0x561858a3f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dd409cb00 tx=0x7f2dd40a1070 comp rx=0 tx=0).stop 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 shutdown_connections 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858a525b0 0x561858a549a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 --2- 172.21.15.7:0/1789265864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a3f750 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 >> 172.21.15.7:0/1789265864 conn(0x561858a338a0 msgr2=0x561858a355f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.689+0000 7f2dc6000640 0 client.25637 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.690+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 shutdown_connections 2024-04-20T21:34:35.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.690+0000 7f2dea053740 1 -- 172.21.15.7:0/1789265864 wait complete. 2024-04-20T21:34:35.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.690+0000 7f2dea053740 20 client.25637 trim_cache size 0 max 16384 2024-04-20T21:34:35.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:35 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1833479004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:35 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1833479004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:35.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 Processor -- start 2024-04-20T21:34:35.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 -- start start 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01b2540 con 0x561858a39a90 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01afe00 con 0x561858a39170 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.718+0000 7f2dea053740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01adc50 con 0x561858a42670 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53204/0 (socket says 172.21.15.7:53204) 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 -- 172.21.15.7:0/2848024328 learned_addr learned my addr 172.21.15.7:0/2848024328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2de4a00640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 -- 172.21.15.7:0/2848024328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 msgr2=0x561858a3c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2de5400640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a3c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 -- 172.21.15.7:0/2848024328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 msgr2=0x561858a41e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a41e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2ddfe00640 1 -- 172.21.15.7:0/2848024328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561858a398c0 con 0x561858a39a90 2024-04-20T21:34:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.719+0000 7f2de5400640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.720+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2de0069d90 tx=0x7f2dcc02cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf2e843811364fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.720+0000 7f2dd1200640 1 -- 172.21.15.7:0/2848024328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc004300 con 0x561858a39a90 2024-04-20T21:34:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.720+0000 7f2dd1200640 1 -- 172.21.15.7:0/2848024328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc0044a0 con 0x561858a39a90 2024-04-20T21:34:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.720+0000 7f2dd1200640 1 -- 172.21.15.7:0/2848024328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0051a0 con 0x561858a39a90 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 -- 172.21.15.7:0/2848024328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 msgr2=0x561858a3ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2de0069d90 tx=0x7f2dcc02cf90 comp rx=0 tx=0).stop 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 -- 172.21.15.7:0/2848024328 shutdown_connections 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a42670 0x561858a3c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39a90 0x561858a3ef00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 --2- 172.21.15.7:0/2848024328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a39170 0x561858a41e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 -- 172.21.15.7:0/2848024328 >> 172.21.15.7:0/2848024328 conn(0x561858a338a0 msgr2=0x561858a35580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 -- 172.21.15.7:0/2848024328 shutdown_connections 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.721+0000 7f2dea053740 1 -- 172.21.15.7:0/2848024328 wait complete. 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 -- start start 2024-04-20T21:34:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 Processor -- start 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858ab98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01ae2c0 con 0x561858a39170 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01af730 con 0x561858a42670 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2de01ae990 con 0x561858a39a90 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.722+0000 7f2dea053740 10 client.0 _ll_register_callbacks cb 0x7f2dd409c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53210/0 (socket says 172.21.15.7:53210) 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de4a00640 1 -- 172.21.15.7:0/2112130345 learned_addr learned my addr 172.21.15.7:0/2112130345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2ddfe00640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858ab98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 -- 172.21.15.7:0/2112130345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 msgr2=0x561858ab98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858ab98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 -- 172.21.15.7:0/2112130345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 msgr2=0x561858ab9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de5400640 1 -- 172.21.15.7:0/2112130345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618586b03f0 con 0x561858a42670 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.723+0000 7f2de4a00640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2de5400640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a548c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009d70 tx=0x7f2dd8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0024044278cb519 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dc6000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8028640 con 0x561858a42670 2024-04-20T21:34:35.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dcc0044a0 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2dd800e3c0 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2dd801fe30 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dea053740 10 client.25643 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.724+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56185877a7d0 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc0044a0 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f2dd8026070 con 0x561858a42670 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 10 client.25643 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 10 client.25643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 20 client.25643 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858ac0ec0 0x561858ac3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 4 client.25643 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:34:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618589a58e0 con 0x561858ac0ec0 2024-04-20T21:34:35.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.725+0000 7f2de4a00640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858ac0ec0 0x561858ac3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2de4a00640 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858ac0ec0 0x561858ac3300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33af78803b634ef6 server_cookie=b77a9ce5c7bdd5e1 in_seq=0 out_seq=0 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dc6000640 10 client.25643 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dc6000640 1 -- 172.21.15.7:0/2112130345 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618589a58e0 con 0x561858ac0ec0 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dc6000640 10 client.25643 handle_command_reply: tid=0 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 client.25643 shutdown 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 msgr2=0x561858a548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a548c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009d70 tx=0x7f2dd8004ab0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 shutdown_connections 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561858a42670 0x561858a548c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561858ac0ec0 0x561858ac3300 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561858a39a90 0x561858ab98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 --2- 172.21.15.7:0/2112130345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561858a39170 0x561858ab9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dc6000640 0 client.25643 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.726+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 >> 172.21.15.7:0/2112130345 conn(0x561858a338a0 msgr2=0x561858a320d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.727+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 shutdown_connections 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.727+0000 7f2dea053740 1 -- 172.21.15.7:0/2112130345 wait complete. 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.727+0000 7f2dea053740 20 client.25643 trim_cache size 0 max 16384 2024-04-20T21:34:35.729 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 msgr2=0x7f2dc80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 0x7f2dc80a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8005fd0 tx=0x7f2dd8009500 comp rx=0 tx=0).stop 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 msgr2=0x7f2de01a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dd400e9f0 tx=0x7f2dd400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 shutdown_connections 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de019a760 0x7f2de019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2de01070a0 0x7f2de01a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2dc80a3f50 0x7f2dc80a6410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 --2- 172.21.15.7:0/379164804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de0101fd0 0x7f2de01a0600 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 >> 172.21.15.7:0/379164804 conn(0x7f2de00fd8b0 msgr2=0x7f2de00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 shutdown_connections 2024-04-20T21:34:35.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:35.728+0000 7f2de5e00640 1 -- 172.21.15.7:0/379164804 wait complete. 2024-04-20T21:34:35.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:35 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1833479004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:34:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:36 smithi016 ceph-mon[21409]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:36 smithi186 ceph-mon[22157]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:36.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:34:36.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:34:36.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:34:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:36 smithi007 ceph-mon[16867]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 -- 172.21.15.7:0/2313030724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4074310 msgr2=0x7f1bb40746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 --2- 172.21.15.7:0/2313030724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4074310 0x7f1bb40746f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009900 tx=0x7f1ba802bcd0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 -- 172.21.15.7:0/2313030724 shutdown_connections 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 --2- 172.21.15.7:0/2313030724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb410e620 0x7f1bb4110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 --2- 172.21.15.7:0/2313030724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 --2- 172.21.15.7:0/2313030724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb4074310 0x7f1bb40746f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 -- 172.21.15.7:0/2313030724 >> 172.21.15.7:0/2313030724 conn(0x7f1bb40fd8b0 msgr2=0x7f1bb40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 -- 172.21.15.7:0/2313030724 shutdown_connections 2024-04-20T21:34:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.941+0000 7f1bbac00640 1 -- 172.21.15.7:0/2313030724 wait complete. 2024-04-20T21:34:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.942+0000 7f1bbac00640 1 Processor -- start 2024-04-20T21:34:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 -- start start 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb410e620 0x7f1bb419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb4114330 con 0x7f1bb4074310 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41141b0 con 0x7f1bb410e620 2024-04-20T21:34:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.943+0000 7f1bbac00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41144b0 con 0x7f1bb4074c30 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53226/0 (socket says 172.21.15.7:53226) 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 -- 172.21.15.7:0/259740604 learned_addr learned my addr 172.21.15.7:0/259740604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bba200640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb410e620 0x7f1bb419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 -- 172.21.15.7:0/259740604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 msgr2=0x7f1bb41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 -- 172.21.15.7:0/259740604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb410e620 msgr2=0x7f1bb419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb410e620 0x7f1bb419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.944+0000 7f1bb9800640 1 -- 172.21.15.7:0/259740604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb419af90 con 0x7f1bb4074310 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bb9800640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005940 tx=0x7f1ba802cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bb2000640 1 -- 172.21.15.7:0/259740604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8041070 con 0x7f1bb4074310 2024-04-20T21:34:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bb2000640 1 -- 172.21.15.7:0/259740604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba8004450 con 0x7f1bb4074310 2024-04-20T21:34:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bb2000640 1 -- 172.21.15.7:0/259740604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8003d80 con 0x7f1bb4074310 2024-04-20T21:34:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.945+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb419b220 con 0x7f1bb4074310 2024-04-20T21:34:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.946+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb41a7590 con 0x7f1bb4074310 2024-04-20T21:34:36.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.950+0000 7f1bbefee740 1 Processor -- start 2024-04-20T21:34:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.950+0000 7f1bbefee740 1 -- start start 2024-04-20T21:34:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 0x55f72d7d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d1a10 0x55f72d7da0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f72d735d20 con 0x55f72d7d10f0 2024-04-20T21:34:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f72d6dc3e0 con 0x55f72d7da5f0 2024-04-20T21:34:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bbefee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f72d731ce0 con 0x55f72d7d1a10 2024-04-20T21:34:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bb8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bb8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45496/0 (socket says 172.21.15.7:45496) 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bb8e00640 1 -- 172.21.15.7:0/3649629417 learned_addr learned my addr 172.21.15.7:0/3649629417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.951+0000 7f1bba200640 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 0x55f72d7d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.952+0000 7f1bb8e00640 1 -- 172.21.15.7:0/3649629417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d1a10 msgr2=0x55f72d7da0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.952+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d1a10 0x55f72d7da0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.952+0000 7f1bb8e00640 1 -- 172.21.15.7:0/3649629417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 msgr2=0x55f72d7d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.952+0000 7f1bb2000640 1 -- 172.21.15.7:0/259740604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f1ba8044070 con 0x7f1bb4074310 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 0x55f72d7d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1bb8e00640 1 -- 172.21.15.7:0/3649629417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72d6fb980 con 0x55f72d7da5f0 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1bba200640 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 0x55f72d7d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00114f0 tx=0x7f1ba00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60739dd30b68ef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1ba6a00640 1 -- 172.21.15.7:0/3649629417 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0004020 con 0x55f72d7da5f0 2024-04-20T21:34:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1ba6a00640 1 -- 172.21.15.7:0/3649629417 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba00041c0 con 0x55f72d7da5f0 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.953+0000 7f1ba6a00640 1 -- 172.21.15.7:0/3649629417 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba000f910 con 0x55f72d7da5f0 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.954+0000 7f1bb2000640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 0x7f1b9c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.954+0000 7f1bb2000640 1 -- 172.21.15.7:0/259740604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ba80f2170 con 0x7f1bb4074310 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.954+0000 7f1bbefee740 1 -- 172.21.15.7:0/3649629417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 msgr2=0x55f72d7dc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.954+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00114f0 tx=0x7f1ba00119b0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bba200640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 0x7f1b9c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bbefee740 1 -- 172.21.15.7:0/3649629417 shutdown_connections 2024-04-20T21:34:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7dc9e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d1a10 0x55f72d7da0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3649629417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d10f0 0x55f72d7d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.955+0000 7f1bbefee740 1 -- 172.21.15.7:0/3649629417 >> 172.21.15.7:0/3649629417 conn(0x55f72d7cb820 msgr2=0x55f72d7cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.956+0000 7f1bba200640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 0x7f1b9c0a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009fd0 tx=0x7f1bac009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.956+0000 7f1bbefee740 1 -- 172.21.15.7:0/3649629417 shutdown_connections 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.956+0000 7f1bbefee740 1 -- 172.21.15.7:0/3649629417 wait complete. 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.956+0000 7f1bbefee740 1 -- start start 2024-04-20T21:34:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 Processor -- start 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7d7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f72d7dd3e0 con 0x55f72d7d1a10 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f72d6ef7a0 con 0x55f72d7da5f0 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.957+0000 7f1bbefee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f72d7dd560 con 0x55f72d7d10f0 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bbefee740 10 client.0 _ll_register_callbacks cb 0x55f72d6f1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45500/0 (socket says 172.21.15.7:45500) 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 -- 172.21.15.7:0/2323077343 learned_addr learned my addr 172.21.15.7:0/2323077343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bb9800640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7d7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 -- 172.21.15.7:0/2323077343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 msgr2=0x55f72d7d7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:36.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7d7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 -- 172.21.15.7:0/2323077343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 msgr2=0x55f72d7d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.958+0000 7f1bba200640 1 -- 172.21.15.7:0/2323077343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72d5107e0 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.959+0000 7f1bba200640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bac003ab0 tx=0x7f1bac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a7aea19cd7d43e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.959+0000 7f1b9b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.959+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004480 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.959+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1bac004620 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.959+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ba00041c0 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.960+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1bac02d020 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.960+0000 7f1bbefee740 10 client.25655 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.960+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f72d570f70 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.960+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba00041c0 con 0x55f72d7da5f0 2024-04-20T21:34:36.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.960+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f1bac032700 con 0x55f72d7da5f0 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 10 client.25655 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 20 client.25655 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d7f31e0 0x55f72d7f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 4 client.25655 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:36.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72d743740 con 0x55f72d7f31e0 2024-04-20T21:34:36.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.961+0000 7f1bb9800640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d7f31e0 0x55f72d7f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:36.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.962+0000 7f1bb9800640 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d7f31e0 0x55f72d7f5610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7458384d4651ae7d server_cookie=9e457915cd0db040 in_seq=0 out_seq=0 2024-04-20T21:34:36.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.962+0000 7f1b9b400640 10 client.25655 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:36.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1b9b400640 1 -- 172.21.15.7:0/2323077343 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f72d743740 con 0x55f72d7f31e0 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1b9b400640 10 client.25655 handle_command_reply: tid=0 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 client.25655 shutdown 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 msgr2=0x55f72d7d8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bac003ab0 tx=0x7f1bac004ab0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 shutdown_connections 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7da5f0 0x55f72d7d8110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d7f31e0 0x55f72d7f5610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 --2- 172.21.15.7:0/2323077343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7d7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1b9b400640 0 client.25655 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 >> 172.21.15.7:0/2323077343 conn(0x55f72d7cb820 msgr2=0x55f72d7cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 shutdown_connections 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.963+0000 7f1bbefee740 1 -- 172.21.15.7:0/2323077343 wait complete. 2024-04-20T21:34:36.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:36.964+0000 7f1bbefee740 20 client.25655 trim_cache size 0 max 16384 2024-04-20T21:34:36.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.000+0000 7f1bbefee740 1 Processor -- start 2024-04-20T21:34:36.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.000+0000 7f1bbefee740 1 -- start start 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 0x55f72d7eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41ae100 con 0x55f72d7d1a10 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41ab9c0 con 0x55f72d7d10f0 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bbefee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41a9830 con 0x55f72d7da5f0 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 0x55f72d7eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bba200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bba200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47204/0 (socket says 172.21.15.7:47204) 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 0x55f72d7eeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45510/0 (socket says 172.21.15.7:45510) 2024-04-20T21:34:37.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bba200640 1 -- 172.21.15.7:0/3571313004 learned_addr learned my addr 172.21.15.7:0/3571313004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.001+0000 7f1bb8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 -- 172.21.15.7:0/3571313004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 msgr2=0x55f72d7eeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 0x55f72d7eeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 -- 172.21.15.7:0/3571313004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 msgr2=0x55f72d7d7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 -- 172.21.15.7:0/3571313004 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72d7d1840 con 0x55f72d7da5f0 2024-04-20T21:34:37.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1bba200640 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bac000f80 tx=0x7f1bac003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc7148e2714b117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.002+0000 7f1ba6000640 1 -- 172.21.15.7:0/3571313004 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004480 con 0x55f72d7da5f0 2024-04-20T21:34:37.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.003+0000 7f1ba6000640 1 -- 172.21.15.7:0/3571313004 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac024030 con 0x55f72d7da5f0 2024-04-20T21:34:37.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.003+0000 7f1ba6000640 1 -- 172.21.15.7:0/3571313004 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00f940 con 0x55f72d7da5f0 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.003+0000 7f1bbefee740 1 -- 172.21.15.7:0/3571313004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 msgr2=0x55f72d7faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.003+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1bac000f80 tx=0x7f1bac003970 comp rx=0 tx=0).stop 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 -- 172.21.15.7:0/3571313004 shutdown_connections 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7da5f0 0x55f72d7faee0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7d1a10 0x55f72d7d7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 --2- 172.21.15.7:0/3571313004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d10f0 0x55f72d7eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 -- 172.21.15.7:0/3571313004 >> 172.21.15.7:0/3571313004 conn(0x55f72d7cb820 msgr2=0x55f72d7caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 -- 172.21.15.7:0/3571313004 shutdown_connections 2024-04-20T21:34:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 -- 172.21.15.7:0/3571313004 wait complete. 2024-04-20T21:34:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.004+0000 7f1bbefee740 1 -- start start 2024-04-20T21:34:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 Processor -- start 2024-04-20T21:34:37.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 0x55f72d7e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 0x55f72d7e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41a9e80 con 0x55f72d7da5f0 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41ab2f0 con 0x55f72d7d1a10 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb41aa550 con 0x55f72d7d10f0 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.005+0000 7f1bbefee740 10 client.0 _ll_register_callbacks cb 0x7f1bac009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47206/0 (socket says 172.21.15.7:47206) 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 -- 172.21.15.7:0/461270118 learned_addr learned my addr 172.21.15.7:0/461270118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bba200640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 0x55f72d7e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 0x55f72d7e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 -- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 msgr2=0x55f72d7e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 0x55f72d7e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 -- 172.21.15.7:0/461270118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 msgr2=0x55f72d7e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 0x55f72d7e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb9800640 1 -- 172.21.15.7:0/461270118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72d408170 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bba200640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 0x55f72d7e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.006+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 0x55f72d7e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1bb9800640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80bdd70 tx=0x7f1ba8009270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4865bd33fd3b9de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1b9b400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba80db020 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ba80c20b0 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bac024030 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1ba80be770 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.007+0000 7f1bbefee740 10 client.25654 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f72d8596a0 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bac024030 con 0x55f72d7d10f0 2024-04-20T21:34:37.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f1ba80b8be0 con 0x55f72d7d10f0 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 10 client.25654 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 20 client.25654 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d859dc0 0x55f72d85c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 4 client.25654 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.008+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f72d73d5b0 con 0x55f72d859dc0 2024-04-20T21:34:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.009+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d859dc0 0x55f72d85c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.010+0000 7f1bb8e00640 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d859dc0 0x55f72d85c1b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09d0018e4729b2c server_cookie=af1280791319cebf in_seq=0 out_seq=0 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.010+0000 7f1b9b400640 10 client.25654 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1b9b400640 1 -- 172.21.15.7:0/461270118 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f72d73d5b0 con 0x55f72d859dc0 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1b9b400640 10 client.25654 handle_command_reply: tid=0 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 client.25654 shutdown 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 msgr2=0x55f72d7eb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ba80bdd70 tx=0x7f1ba8009270 comp rx=0 tx=0).stop 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 shutdown_connections 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f72d7d1a10 0x55f72d7e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.011+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f72d859dc0 0x55f72d85c1b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f72d7da5f0 0x55f72d7e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1b9b400640 0 client.25654 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 1 --2- 172.21.15.7:0/461270118 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f72d7d10f0 0x55f72d7eb480 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 >> 172.21.15.7:0/461270118 conn(0x55f72d7cb820 msgr2=0x55f72d7cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 shutdown_connections 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 1 -- 172.21.15.7:0/461270118 wait complete. 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.012+0000 7f1bbefee740 20 client.25654 trim_cache size 0 max 16384 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:37.012 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:34:37.013 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "77ccd162-7a8a-4087-ba8c-2f1aa62e3639", 2024-04-20T21:34:37.013 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:34:37.013 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 msgr2=0x7f1b9c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 0x7f1b9c0a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009fd0 tx=0x7f1bac009500 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 msgr2=0x7f1bb41a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005940 tx=0x7f1ba802cbd0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 shutdown_connections 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b9c0a3f50 0x7f1b9c0a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bb410e620 0x7f1bb419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1bb4074c30 0x7f1bb41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 --2- 172.21.15.7:0/259740604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb4074310 0x7f1bb41a0610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.013+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 >> 172.21.15.7:0/259740604 conn(0x7f1bb40fd8b0 msgr2=0x7f1bb40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.014+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 shutdown_connections 2024-04-20T21:34:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.014+0000 7f1bbac00640 1 -- 172.21.15.7:0/259740604 wait complete. 2024-04-20T21:34:37.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:34:37.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 -- 172.21.15.7:0/164446150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 msgr2=0x7f1630103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 --2- 172.21.15.7:0/164446150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1620009860 tx=0x7f162002bc20 comp rx=0 tx=0).stop 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 -- 172.21.15.7:0/164446150 shutdown_connections 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 --2- 172.21.15.7:0/164446150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1630103860 0x7f163010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 --2- 172.21.15.7:0/164446150 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f163010c360 0x7f163010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 --2- 172.21.15.7:0/164446150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 -- 172.21.15.7:0/164446150 >> 172.21.15.7:0/164446150 conn(0x7f1630077a10 msgr2=0x7f16300fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 -- 172.21.15.7:0/164446150 shutdown_connections 2024-04-20T21:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.356+0000 7f162f200640 1 -- 172.21.15.7:0/164446150 wait complete. 2024-04-20T21:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.358+0000 7f162f200640 1 Processor -- start 2024-04-20T21:34:37.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.358+0000 7f162f200640 1 -- start start 2024-04-20T21:34:37.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1630103860 0x7f16301042f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f163010c360 0x7f1630104830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1630112110 con 0x7f1630102f40 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1630111f90 con 0x7f163010c360 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1630112290 con 0x7f1630103860 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53278/0 (socket says 172.21.15.7:53278) 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1630103860 0x7f16301042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162de00640 1 -- 172.21.15.7:0/1381416361 learned_addr learned my addr 172.21.15.7:0/1381416361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.359+0000 7f162e800640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f163010c360 0x7f1630104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 -- 172.21.15.7:0/1381416361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1630103860 msgr2=0x7f16301042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1630103860 0x7f16301042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 -- 172.21.15.7:0/1381416361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f163010c360 msgr2=0x7f1630104830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f163010c360 0x7f1630104830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 -- 172.21.15.7:0/1381416361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301aaf80 con 0x7f1630102f40 2024-04-20T21:34:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.360+0000 7f162de00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1620005e00 tx=0x7f16200344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.361+0000 7f1626a00640 1 -- 172.21.15.7:0/1381416361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620041020 con 0x7f1630102f40 2024-04-20T21:34:37.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.361+0000 7f1626a00640 1 -- 172.21.15.7:0/1381416361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f162002ce50 con 0x7f1630102f40 2024-04-20T21:34:37.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.361+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16301ab210 con 0x7f1630102f40 2024-04-20T21:34:37.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.361+0000 7f1626a00640 1 -- 172.21.15.7:0/1381416361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620039040 con 0x7f1630102f40 2024-04-20T21:34:37.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.361+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301ab6b0 con 0x7f1630102f40 2024-04-20T21:34:37.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.366+0000 7f163799e740 1 Processor -- start 2024-04-20T21:34:37.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.366+0000 7f163799e740 1 -- start start 2024-04-20T21:34:37.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf33340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 0x55e8ecf35c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f1626a00640 1 -- 172.21.15.7:0/1381416361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 26) ==== 138929+0+0 (secure 0 0 0) 0x7f1620044020 con 0x7f1630102f40 2024-04-20T21:34:37.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8ece90150 con 0x55e8ecf2aca0 2024-04-20T21:34:37.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ece36370 con 0x55e8ecf2a380 2024-04-20T21:34:37.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f163799e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8ece8c110 con 0x55e8ecf33880 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45550/0 (socket says 172.21.15.7:45550) 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.367+0000 7f162e800640 1 -- 172.21.15.7:0/1466134988 learned_addr learned my addr 172.21.15.7:0/1466134988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162d400640 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 0x55e8ecf35c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162e800640 1 -- 172.21.15.7:0/1466134988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 msgr2=0x55e8ecf35c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162e800640 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 0x55e8ecf35c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162e800640 1 -- 172.21.15.7:0/1466134988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 msgr2=0x55e8ecf33340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162e800640 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162e800640 1 -- 172.21.15.7:0/1466134988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ecc6c7d0 con 0x55e8ecf2a380 2024-04-20T21:34:37.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.368+0000 7f162d400640 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 0x55e8ecf35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.369+0000 7f162e800640 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1628009f50 tx=0x7f16280118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e5688a44cf176e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.369+0000 7f161b000640 1 -- 172.21.15.7:0/1466134988 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628004d20 con 0x55e8ecf2a380 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.369+0000 7f161b000640 1 -- 172.21.15.7:0/1466134988 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1628004050 con 0x55e8ecf2a380 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.369+0000 7f161b000640 1 -- 172.21.15.7:0/1466134988 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628005260 con 0x55e8ecf2a380 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.370+0000 7f1626a00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 0x7f16140a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.370+0000 7f1626a00640 1 -- 172.21.15.7:0/1381416361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f16200f1f30 con 0x7f1630102f40 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.370+0000 7f162de00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 0x7f16140a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.370+0000 7f163799e740 1 -- 172.21.15.7:0/1466134988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 msgr2=0x55e8ecf2a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.370+0000 7f163799e740 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1628009f50 tx=0x7f16280118d0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 -- 172.21.15.7:0/1466134988 shutdown_connections 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf33880 0x55e8ecf35c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf33340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 --2- 172.21.15.7:0/1466134988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf2a760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 -- 172.21.15.7:0/1466134988 >> 172.21.15.7:0/1466134988 conn(0x55e8ecf24ab0 msgr2=0x55e8ecf24e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 -- 172.21.15.7:0/1466134988 shutdown_connections 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f163799e740 1 -- 172.21.15.7:0/1466134988 wait complete. 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.371+0000 7f162de00640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 0x7f16140a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16200ef600 tx=0x7f16200de040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.372+0000 7f163799e740 1 -- start start 2024-04-20T21:34:37.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.372+0000 7f163799e740 1 Processor -- start 2024-04-20T21:34:37.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8ecf36670 con 0x55e8ecf33880 2024-04-20T21:34:37.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8ece49740 con 0x55e8ecf2a380 2024-04-20T21:34:37.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8ecf367f0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f163799e740 10 client.0 _ll_register_callbacks cb 0x55e8ece44b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f162d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf30870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45552/0 (socket says 172.21.15.7:45552) 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f162e800640 1 -- 172.21.15.7:0/3173853122 learned_addr learned my addr 172.21.15.7:0/3173853122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.373+0000 7f162e800640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 -- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 msgr2=0x55e8ecf30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 -- 172.21.15.7:0/3173853122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 msgr2=0x55e8ecf312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 -- 172.21.15.7:0/3173853122 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ecba23f0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162e800640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:37.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f162d400640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf30db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1630105570 tx=0x7f161c011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3833265a6b9dae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.374+0000 7f160f400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.375+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c01e060 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.375+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f161c00fbc0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.375+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1628004050 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f161c00fdc0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 10 client.25666 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8ecb620f0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f161c0047a0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f161c019070 con 0x55e8ecf2aca0 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 10 client.25666 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 20 client.25666 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecf43c60 0x55e8ecf46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 4 client.25666 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.376+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1628004050 con 0x55e8ecf43c60 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.377+0000 7f162e800640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecf43c60 0x55e8ecf46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.378+0000 7f162e800640 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecf43c60 0x55e8ecf46080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1397709029bb03 server_cookie=52a6ff2e58dbea52 in_seq=0 out_seq=0 2024-04-20T21:34:37.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.378+0000 7f160f400640 10 client.25666 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.378+0000 7f160f400640 1 -- 172.21.15.7:0/3173853122 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1628004050 con 0x55e8ecf43c60 2024-04-20T21:34:37.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.378+0000 7f160f400640 10 client.25666 handle_command_reply: tid=0 2024-04-20T21:34:37.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.378+0000 7f163799e740 1 client.25666 shutdown 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 msgr2=0x55e8ecf30db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf30db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1630105570 tx=0x7f161c011b30 comp rx=0 tx=0).stop 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 shutdown_connections 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecf43c60 0x55e8ecf46080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf30db0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 --2- 172.21.15.7:0/3173853122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 >> 172.21.15.7:0/3173853122 conn(0x55e8ecf24ab0 msgr2=0x55e8ecf26800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f160f400640 0 client.25666 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 shutdown_connections 2024-04-20T21:34:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.379+0000 7f163799e740 1 -- 172.21.15.7:0/3173853122 wait complete. 2024-04-20T21:34:37.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.380+0000 7f163799e740 20 client.25666 trim_cache size 0 max 16384 2024-04-20T21:34:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.416+0000 7f163799e740 1 Processor -- start 2024-04-20T21:34:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.416+0000 7f163799e740 1 -- start start 2024-04-20T21:34:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2a380 0x55e8ecf32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b6900 con 0x55e8ecf2aca0 2024-04-20T21:34:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b41c0 con 0x55e8ecf33880 2024-04-20T21:34:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f163799e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b2010 con 0x55e8ecf2a380 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45560/0 (socket says 172.21.15.7:45560) 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f162e800640 1 -- 172.21.15.7:0/1249204426 learned_addr learned my addr 172.21.15.7:0/1249204426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f162e800640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2a380 0x55e8ecf32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.417+0000 7f162d400640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 -- 172.21.15.7:0/1249204426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2a380 msgr2=0x55e8ecf32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2a380 0x55e8ecf32fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 -- 172.21.15.7:0/1249204426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 msgr2=0x55e8ecf3f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 -- 172.21.15.7:0/1249204426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ecf35c70 con 0x55e8ecf33880 2024-04-20T21:34:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162d400640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.418+0000 7f162e800640 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1628009270 tx=0x7f1628009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcf1850034f5be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.419+0000 7f161a600640 1 -- 172.21.15.7:0/1249204426 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628004bd0 con 0x55e8ecf33880 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.419+0000 7f161a600640 1 -- 172.21.15.7:0/1249204426 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1628004d70 con 0x55e8ecf33880 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.419+0000 7f161a600640 1 -- 172.21.15.7:0/1249204426 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628005290 con 0x55e8ecf33880 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 -- 172.21.15.7:0/1249204426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 msgr2=0x55e8ecf4b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1628009270 tx=0x7f1628009500 comp rx=0 tx=0).stop 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 -- 172.21.15.7:0/1249204426 shutdown_connections 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf33880 0x55e8ecf4b950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 --2- 172.21.15.7:0/1249204426 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2a380 0x55e8ecf32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 -- 172.21.15.7:0/1249204426 >> 172.21.15.7:0/1249204426 conn(0x55e8ecf24ab0 msgr2=0x55e8ecf24e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 -- 172.21.15.7:0/1249204426 shutdown_connections 2024-04-20T21:34:37.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.420+0000 7f163799e740 1 -- 172.21.15.7:0/1249204426 wait complete. 2024-04-20T21:34:37.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.421+0000 7f163799e740 1 -- start start 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.421+0000 7f163799e740 1 Processor -- start 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b2680 con 0x55e8ecf33880 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b3af0 con 0x55e8ecf2a380 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f16301b2d50 con 0x55e8ecf2aca0 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f163799e740 10 client.0 _ll_register_callbacks cb 0x7f161c009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f162d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f162e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f162d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47262/0 (socket says 172.21.15.7:47262) 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f162d400640 1 -- 172.21.15.7:0/190993324 learned_addr learned my addr 172.21.15.7:0/190993324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:34:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.422+0000 7f162e800640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 -- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 msgr2=0x55e8ecf3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 -- 172.21.15.7:0/190993324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 msgr2=0x55e8ecf36ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf36ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 -- 172.21.15.7:0/190993324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ecf377f0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162e800640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f162d400640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55e8ecf300d0 tx=0x7f161c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be76e824672c989c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.423+0000 7f160f400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c000e60 con 0x55e8ecf2aca0 2024-04-20T21:34:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f161c00fbc0 con 0x55e8ecf2aca0 2024-04-20T21:34:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1628004d70 con 0x55e8ecf2aca0 2024-04-20T21:34:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f161c03b050 con 0x55e8ecf2aca0 2024-04-20T21:34:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f163799e740 10 client.25672 fetch_fsmap learned FSMap version 24 2024-04-20T21:34:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.424+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e8ecfa9e50 con 0x55e8ecf2aca0 2024-04-20T21:34:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1628004d70 con 0x55e8ecf2aca0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 24) ==== 8339+0+0 (secure 0 0 0) 0x7f161c019040 con 0x55e8ecf2aca0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 10 client.25672 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 20 client.25672 populate_metadata read hostname 'smithi007' 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecfaa5a0 0x55e8ecfac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 4 client.25672 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8ece9db70 con 0x55e8ecfaa5a0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.425+0000 7f162e800640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecfaa5a0 0x55e8ecfac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.426+0000 7f162e800640 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecfaa5a0 0x55e8ecfac9c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecddfcd4908f9807 server_cookie=1d223fddf2956fb5 in_seq=0 out_seq=0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.426+0000 7f160f400640 10 client.25672 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.426+0000 7f160f400640 1 -- 172.21.15.7:0/190993324 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e8ece9db70 con 0x55e8ecfaa5a0 2024-04-20T21:34:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.426+0000 7f160f400640 10 client.25672 handle_command_reply: tid=0 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.426+0000 7f163799e740 1 client.25672 shutdown 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 msgr2=0x55e8ecf3bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55e8ecf300d0 tx=0x7f161c009f90 comp rx=0 tx=0).stop 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 shutdown_connections 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8ecfaa5a0 0x55e8ecfac9c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8ecf33880 0x55e8ecf36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8ecf2aca0 0x55e8ecf3bf90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 --2- 172.21.15.7:0/190993324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8ecf2a380 0x55e8ecf3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 >> 172.21.15.7:0/190993324 conn(0x55e8ecf24ab0 msgr2=0x55e8ecf2bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f160f400640 0 client.25672 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 shutdown_connections 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 1 -- 172.21.15.7:0/190993324 wait complete. 2024-04-20T21:34:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.427+0000 7f163799e740 20 client.25672 trim_cache size 0 max 16384 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "77ccd162-7a8a-4087-ba8c-2f1aa62e3639": { 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "77ccd162-7a8a-4087-ba8c-2f1aa62e3639", 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:34:37.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:34:37.429 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:34:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 msgr2=0x7f16140a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 0x7f16140a6410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16200ef600 tx=0x7f16200de040 comp rx=0 tx=0).stop 2024-04-20T21:34:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 msgr2=0x7f1630103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:34:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1620005e00 tx=0x7f16200344c0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 shutdown_connections 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f16140a3f50 0x7f16140a6410 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f163010c360 0x7f1630104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1630103860 0x7f16301042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 --2- 172.21.15.7:0/1381416361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1630102f40 0x7f1630103db0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.428+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 >> 172.21.15.7:0/1381416361 conn(0x7f1630077a10 msgr2=0x7f16300fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.429+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 shutdown_connections 2024-04-20T21:34:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:34:37.429+0000 7f162f200640 1 -- 172.21.15.7:0/1381416361 wait complete. 2024-04-20T21:34:37.452 INFO:tasks.cephfs.filesystem:scrub status for tag:77ccd162-7a8a-4087-ba8c-2f1aa62e3639 - {'path': '/', 'tag': '77ccd162-7a8a-4087-ba8c-2f1aa62e3639', 'options': 'recursive,force'} 2024-04-20T21:34:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:38 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:34:38.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:38 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:38 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:34:38.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:38 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:38 smithi016 ceph-mon[21409]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-20T21:34:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:38 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:34:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:38 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:38 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:34:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:38 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:38 smithi186 ceph-mon[22157]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-20T21:34:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:38 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:34:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:38 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:38 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:34:38.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:38 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:34:38.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:38 smithi007 ceph-mon[16867]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-20T21:34:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:40 smithi016 ceph-mon[21409]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:40 smithi186 ceph-mon[22157]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:40.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:40 smithi007 ceph-mon[16867]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:42 smithi016 ceph-mon[21409]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:42.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:42 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:34:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:42 smithi186 ceph-mon[22157]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:42 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:34:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:42 smithi007 ceph-mon[16867]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:42 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:34:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:44 smithi016 ceph-mon[21409]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:44 smithi186 ceph-mon[22157]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:44.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:44 smithi007 ceph-mon[16867]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:46.698 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:46 smithi016 ceph-mon[21409]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:46 smithi186 ceph-mon[22157]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:46.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:46 smithi007 ceph-mon[16867]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-20T21:34:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:48 smithi016 ceph-mon[21409]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:48 smithi186 ceph-mon[22157]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:48 smithi007 ceph-mon[16867]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-20T21:34:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:50 smithi016 ceph-mon[21409]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:34:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:50 smithi186 ceph-mon[22157]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:34:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:50 smithi007 ceph-mon[16867]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:34:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:52 smithi016 ceph-mon[21409]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:52 smithi186 ceph-mon[22157]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:52 smithi007 ceph-mon[16867]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:34:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:54 smithi016 ceph-mon[21409]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:34:54.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:54 smithi016 ceph-mon[21409]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T21:34:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:54 smithi186 ceph-mon[22157]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:34:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:54 smithi186 ceph-mon[22157]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T21:34:54.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:54 smithi007 ceph-mon[16867]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:34:54.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:54 smithi007 ceph-mon[16867]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-04-20T21:34:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:56 smithi016 ceph-mon[21409]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:34:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:56 smithi186 ceph-mon[22157]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:34:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:56 smithi007 ceph-mon[16867]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:34:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:34:58 smithi016 ceph-mon[21409]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:34:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:34:58 smithi186 ceph-mon[22157]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:34:58.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:34:58 smithi007 ceph-mon[16867]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-20T21:35:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:00 smithi016 ceph-mon[21409]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:00 smithi186 ceph-mon[22157]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:00 smithi007 ceph-mon[16867]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:01.007 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 58% (7162/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 94% (11637/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-20T21:35:01.586 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to 'a8c84f5997950fd7d008204839253291d52e6eda'. 2024-04-20T21:35:01.586 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.586 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2024-04-20T21:35:01.587 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.588 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-20T21:35:01.588 INFO:tasks.workunit.client.0.smithi007.stderr: 2024-04-20T21:35:01.588 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at a8c84f59979 Merge PR #51659 into wip-rishabh-testing-20240416.193735 2024-04-20T21:35:01.601 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-20T21:35:01.664 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-20T21:35:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:35:01.671 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-20T21:35:02.451 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-20T21:35:02.576 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-20T21:35:02.667 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-20T21:35:02.674 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:35:02.674 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-20T21:35:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:02 smithi016 ceph-mon[21409]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:02 smithi186 ceph-mon[22157]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:02.762 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-20T21:35:02.773 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:35:02.774 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-20T21:35:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:02 smithi007 ceph-mon[16867]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:35:02.824 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-20T21:35:02.826 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-20T21:35:02.826 DEBUG:teuthology.orchestra.run.smithi007:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-20T21:35:02.947 INFO:tasks.workunit.client.0.smithi007.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-20T21:35:02.950 INFO:tasks.workunit.client.0.smithi007.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-20T21:35:02.950 INFO:tasks.workunit.client.0.smithi007.stderr:+ trap -- retry EXIT 2024-04-20T21:35:02.950 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm -rf ffsb 2024-04-20T21:35:02.953 INFO:tasks.workunit.client.0.smithi007.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-20T21:35:02.963 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into 'ffsb'... 2024-04-20T21:35:04.341 INFO:tasks.workunit.client.0.smithi007.stderr:+ trap - EXIT 2024-04-20T21:35:04.341 INFO:tasks.workunit.client.0.smithi007.stderr:+ cd ffsb 2024-04-20T21:35:04.341 INFO:tasks.workunit.client.0.smithi007.stderr:+ ./configure 2024-04-20T21:35:04.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:04 smithi016 ceph-mon[21409]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:35:04.698 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-20T21:35:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:04 smithi186 ceph-mon[22157]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:35:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:04 smithi007 ceph-mon[16867]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-20T21:35:05.718 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether build environment is sane... yes 2024-04-20T21:35:05.745 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-20T21:35:05.746 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gawk... gawk 2024-04-20T21:35:05.778 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether make sets $(MAKE)... yes 2024-04-20T21:35:05.881 INFO:tasks.workunit.client.0.smithi007.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-20T21:35:05.881 INFO:tasks.workunit.client.0.smithi007.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-20T21:35:05.902 INFO:tasks.workunit.client.0.smithi007.stdout:checking for style of include used by make... GNU 2024-04-20T21:35:05.908 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... gcc 2024-04-20T21:35:06.057 INFO:tasks.workunit.client.0.smithi007.stdout:checking for C compiler default output file name... a.out 2024-04-20T21:35:06.064 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether the C compiler works... yes 2024-04-20T21:35:06.069 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are cross compiling... no 2024-04-20T21:35:06.148 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of executables... 2024-04-20T21:35:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of object files... o 2024-04-20T21:35:06.314 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... yes 2024-04-20T21:35:06.394 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... yes 2024-04-20T21:35:06.545 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-20T21:35:06.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:06 smithi016 ceph-mon[21409]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-20T21:35:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:06 smithi186 ceph-mon[22157]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-20T21:35:06.759 INFO:tasks.workunit.client.0.smithi007.stdout:checking dependency style of gcc... gcc3 2024-04-20T21:35:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:06 smithi007 ceph-mon[16867]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-20T21:35:06.933 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to run the C preprocessor... gcc -E 2024-04-20T21:35:07.070 INFO:tasks.workunit.client.0.smithi007.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-20T21:35:07.075 INFO:tasks.workunit.client.0.smithi007.stdout:checking for egrep... /usr/bin/grep -E 2024-04-20T21:35:07.105 INFO:tasks.workunit.client.0.smithi007.stdout:checking for AIX... no 2024-04-20T21:35:07.129 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... (cached) gcc 2024-04-20T21:35:07.162 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-20T21:35:07.165 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-20T21:35:07.166 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-20T21:35:07.166 INFO:tasks.workunit.client.0.smithi007.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-20T21:35:07.315 INFO:tasks.workunit.client.0.smithi007.stdout:checking for main in -lm... yes 2024-04-20T21:35:07.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:07.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:07.453 INFO:tasks.workunit.client.0.smithi007.stdout:checking for main in -lpthread... yes 2024-04-20T21:35:07.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.657+0000 7f69bf000640 1 -- 172.21.15.7:0/3288040601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b810e6e0 msgr2=0x7f69b8110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.657+0000 7f69bf000640 1 --2- 172.21.15.7:0/3288040601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b810e6e0 0x7f69b8110ad0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009ca0 tx=0x7f69b002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 -- 172.21.15.7:0/3288040601 shutdown_connections 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 --2- 172.21.15.7:0/3288040601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b810e6e0 0x7f69b8110ad0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 --2- 172.21.15.7:0/3288040601 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b81014e0 0x7f69b810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 --2- 172.21.15.7:0/3288040601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8100b30 0x7f69b8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 -- 172.21.15.7:0/3288040601 >> 172.21.15.7:0/3288040601 conn(0x7f69b80fc7a0 msgr2=0x7f69b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 -- 172.21.15.7:0/3288040601 shutdown_connections 2024-04-20T21:35:07.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.658+0000 7f69bf000640 1 -- 172.21.15.7:0/3288040601 wait complete. 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 Processor -- start 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 -- start start 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b8100b30 0x7f69b81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 0x7f69b819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8114330 con 0x7f69b810e6e0 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81141b0 con 0x7f69b81014e0 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.659+0000 7f69bf000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81144b0 con 0x7f69b8100b30 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40544/0 (socket says 172.21.15.7:40544) 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 -- 172.21.15.7:0/1329873364 learned_addr learned my addr 172.21.15.7:0/1329873364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b8100b30 0x7f69b81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69be600640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 0x7f69b819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 -- 172.21.15.7:0/1329873364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b8100b30 msgr2=0x7f69b81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b8100b30 0x7f69b81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 -- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 msgr2=0x7f69b819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 0x7f69b819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 -- 172.21.15.7:0/1329873364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b819af90 con 0x7f69b81014e0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69be600640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 0x7f69b819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69bd200640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9f0 tx=0x7f69a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.660+0000 7f69b6000640 1 -- 172.21.15.7:0/1329873364 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a400cd80 con 0x7f69b81014e0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.661+0000 7f69b6000640 1 -- 172.21.15.7:0/1329873364 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a4005500 con 0x7f69b81014e0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.661+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b819b280 con 0x7f69b81014e0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.661+0000 7f69b6000640 1 -- 172.21.15.7:0/1329873364 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4010690 con 0x7f69b81014e0 2024-04-20T21:35:07.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.661+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a7590 con 0x7f69b81014e0 2024-04-20T21:35:07.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.664+0000 7f69c342a740 1 Processor -- start 2024-04-20T21:35:07.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.664+0000 7f69c342a740 1 -- start start 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69b6000640 1 -- 172.21.15.7:0/1329873364 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f69a4010830 con 0x7f69b81014e0 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 0x561b80428760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80431880 0x561b80433c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561b8038e150 con 0x561b80428380 2024-04-20T21:35:07.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561b80334370 con 0x561b80428ca0 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69c342a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561b8038a110 con 0x561b80431880 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40552/0 (socket says 172.21.15.7:40552) 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 -- 172.21.15.7:0/4046951216 learned_addr learned my addr 172.21.15.7:0/4046951216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bd200640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80431880 0x561b80433c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69be600640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 0x561b80428760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 -- 172.21.15.7:0/4046951216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80431880 msgr2=0x561b80433c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80431880 0x561b80433c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 -- 172.21.15.7:0/4046951216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 msgr2=0x561b80428760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 0x561b80428760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69bdc00640 1 -- 172.21.15.7:0/4046951216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b8016a7d0 con 0x561b80428ca0 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.665+0000 7f69be600640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 0x561b80428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:07.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.666+0000 7f69bdc00640 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009cf0 tx=0x7f69ac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3ea6860554077d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.666+0000 7f69aaa00640 1 -- 172.21.15.7:0/4046951216 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac004230 con 0x561b80428ca0 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.666+0000 7f69aaa00640 1 -- 172.21.15.7:0/4046951216 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ac0043d0 con 0x561b80428ca0 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.666+0000 7f69aaa00640 1 -- 172.21.15.7:0/4046951216 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac019900 con 0x561b80428ca0 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69b6000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 0x7f69a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 -- 172.21.15.7:0/4046951216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 msgr2=0x561b80431340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009cf0 tx=0x7f69ac011810 comp rx=0 tx=0).stop 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69b6000640 1 -- 172.21.15.7:0/1329873364 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f69a40d0c30 con 0x7f69b81014e0 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 -- 172.21.15.7:0/4046951216 shutdown_connections 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b80431340 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80431880 0x561b80433c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 --2- 172.21.15.7:0/4046951216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80428380 0x561b80428760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 -- 172.21.15.7:0/4046951216 >> 172.21.15.7:0/4046951216 conn(0x561b80422ab0 msgr2=0x561b80422e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69be600640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 0x7f69a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 -- 172.21.15.7:0/4046951216 shutdown_connections 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.667+0000 7f69c342a740 1 -- 172.21.15.7:0/4046951216 wait complete. 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.668+0000 7f69c342a740 1 -- start start 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.668+0000 7f69be600640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 0x7f69a00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005e30 tx=0x7f69b0005d80 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.668+0000 7f69c342a740 1 Processor -- start 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b8042e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561b80434670 con 0x561b80431880 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561b80347740 con 0x561b80428ca0 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561b804347f0 con 0x561b80428380 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69c342a740 10 client.0 _ll_register_callbacks cb 0x561b80342b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40554/0 (socket says 172.21.15.7:40554) 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 -- 172.21.15.7:0/920022652 learned_addr learned my addr 172.21.15.7:0/920022652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bdc00640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b8042e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69be600640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 -- 172.21.15.7:0/920022652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 msgr2=0x561b8042e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b8042e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 -- 172.21.15.7:0/920022652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 msgr2=0x561b8042f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bd200640 1 -- 172.21.15.7:0/920022652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b800a03f0 con 0x561b80428ca0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.669+0000 7f69bdc00640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b8042e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f69bd200640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9c0 tx=0x7f69a4004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8c75aa9ed3040b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f699f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a409c560 con 0x561b80428ca0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f69a40bd0b0 con 0x561b80428ca0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69ac0043d0 con 0x561b80428ca0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69a409d300 con 0x561b80428ca0 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f69c342a740 10 client.25688 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:07.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b800600f0 con 0x561b80428ca0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.670+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69ac0043d0 con 0x561b80428ca0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f69a40ce630 con 0x561b80428ca0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 10 client.25688 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 20 client.25688 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b80441bb0 0x561b80444050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 4 client.25688 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8039bb70 con 0x561b80441bb0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.671+0000 7f69bdc00640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b80441bb0 0x561b80444050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.672+0000 7f69bdc00640 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b80441bb0 0x561b80444050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b233987ee9f5d5b1 server_cookie=2a287f4ae8d8db39 in_seq=0 out_seq=0 2024-04-20T21:35:07.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.672+0000 7f699f400640 10 client.25688 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:07.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.672+0000 7f699f400640 1 -- 172.21.15.7:0/920022652 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b8039bb70 con 0x561b80441bb0 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.672+0000 7f699f400640 10 client.25688 handle_command_reply: tid=0 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.672+0000 7f69c342a740 1 client.25688 shutdown 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 msgr2=0x561b8042edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9c0 tx=0x7f69a4004ae0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 shutdown_connections 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8042edf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b80441bb0 0x561b80444050 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 --2- 172.21.15.7:0/920022652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b8042e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 >> 172.21.15.7:0/920022652 conn(0x561b80422ab0 msgr2=0x561b80424800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f699f400640 0 client.25688 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 shutdown_connections 2024-04-20T21:35:07.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 1 -- 172.21.15.7:0/920022652 wait complete. 2024-04-20T21:35:07.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.673+0000 7f69c342a740 20 client.25688 trim_cache size 0 max 16384 2024-04-20T21:35:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 Processor -- start 2024-04-20T21:35:07.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 -- start start 2024-04-20T21:35:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b80430fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81b25c0 con 0x561b80431880 2024-04-20T21:35:07.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.692+0000 7f69c342a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81afe80 con 0x561b80428ca0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69c342a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81adcd0 con 0x561b80428380 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40570/0 (socket says 172.21.15.7:40570) 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 -- 172.21.15.7:0/3464239712 learned_addr learned my addr 172.21.15.7:0/3464239712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bdc00640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b80430fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69be600640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 -- 172.21.15.7:0/3464239712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 msgr2=0x561b80430fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b80430fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 -- 172.21.15.7:0/3464239712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 msgr2=0x561b8042ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bd200640 1 -- 172.21.15.7:0/3464239712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b80433c70 con 0x561b80428ca0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.693+0000 7f69bdc00640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b80430fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69bd200640 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c6a0 tx=0x7f69a40d47b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2475a6c1be57e121 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69aa000640 1 -- 172.21.15.7:0/3464239712 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a409c560 con 0x561b80428ca0 2024-04-20T21:35:07.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69aa000640 1 -- 172.21.15.7:0/3464239712 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a409cc00 con 0x561b80428ca0 2024-04-20T21:35:07.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69aa000640 1 -- 172.21.15.7:0/3464239712 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4098ea0 con 0x561b80428ca0 2024-04-20T21:35:07.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69c342a740 1 -- 172.21.15.7:0/3464239712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 msgr2=0x561b8043d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.694+0000 7f69c342a740 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c6a0 tx=0x7f69a40d47b0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 -- 172.21.15.7:0/3464239712 shutdown_connections 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428ca0 0x561b8043d530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 --2- 172.21.15.7:0/3464239712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428380 0x561b80430fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 -- 172.21.15.7:0/3464239712 >> 172.21.15.7:0/3464239712 conn(0x561b80422ab0 msgr2=0x561b8042e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 -- 172.21.15.7:0/3464239712 shutdown_connections 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 -- 172.21.15.7:0/3464239712 wait complete. 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 -- start start 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.695+0000 7f69c342a740 1 Processor -- start 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428ca0 0x561b8042b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81ae340 con 0x561b80431880 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81af7b0 con 0x561b80428380 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f69b81aea10 con 0x561b80428ca0 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69c342a740 10 client.0 _ll_register_callbacks cb 0x7f69a40d4410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40576/0 (socket says 172.21.15.7:40576) 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 -- 172.21.15.7:0/1127635982 learned_addr learned my addr 172.21.15.7:0/1127635982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bd200640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428ca0 0x561b8042b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 -- 172.21.15.7:0/1127635982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428ca0 msgr2=0x561b8042b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428ca0 0x561b8042b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 -- 172.21.15.7:0/1127635982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 msgr2=0x561b8042bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.696+0000 7f69bdc00640 1 -- 172.21.15.7:0/1127635982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b804a7190 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f69bdc00640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009250 tx=0x7f69ac009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ec05837d0e376d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f699f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac02bd50 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f69ac005040 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a409cc00 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f69ac037020 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f69c342a740 10 client.25694 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.697+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561b804a8340 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69a409cc00 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f69ac0035f0 con 0x561b80428380 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 10 client.25694 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:07.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 20 client.25694 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b804a8930 0x561b804aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 4 client.25694 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b803959e0 con 0x561b804a8930 2024-04-20T21:35:07.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.698+0000 7f69bd200640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b804a8930 0x561b804aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69bd200640 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b804a8930 0x561b804aad50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22769942a6b5fb5f server_cookie=1109d5b8753583f6 in_seq=0 out_seq=0 2024-04-20T21:35:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f699f400640 10 client.25694 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f699f400640 1 -- 172.21.15.7:0/1127635982 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561b803959e0 con 0x561b804a8930 2024-04-20T21:35:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f699f400640 10 client.25694 handle_command_reply: tid=0 2024-04-20T21:35:07.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 client.25694 shutdown 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 msgr2=0x561b80439fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009250 tx=0x7f69ac009fa0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 shutdown_connections 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561b804a8930 0x561b804aad50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561b80431880 0x561b8042bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561b80428ca0 0x561b8042b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 --2- 172.21.15.7:0/1127635982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561b80428380 0x561b80439fb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 >> 172.21.15.7:0/1127635982 conn(0x561b80422ab0 msgr2=0x561b8042f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f699f400640 0 client.25694 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 shutdown_connections 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 1 -- 172.21.15.7:0/1127635982 wait complete. 2024-04-20T21:35:07.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.699+0000 7f69c342a740 20 client.25694 trim_cache size 0 max 16384 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 msgr2=0x7f69a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 0x7f69a00a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f69b0005e30 tx=0x7f69b0005d80 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 msgr2=0x7f69b81a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69a400e9f0 tx=0x7f69a400edf0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 shutdown_connections 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69a00a3d20 0x7f69a00a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69b81014e0 0x7f69b81a0b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b810e6e0 0x7f69b819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 --2- 172.21.15.7:0/1329873364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f69b8100b30 0x7f69b81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.700+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 >> 172.21.15.7:0/1329873364 conn(0x7f69b80fc7a0 msgr2=0x7f69b80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.701+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 shutdown_connections 2024-04-20T21:35:07.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.701+0000 7f69bf000640 1 -- 172.21.15.7:0/1329873364 wait complete. 2024-04-20T21:35:07.717 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:35:07.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:07.777 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ANSI C header files... yes 2024-04-20T21:35:07.880 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 -- 172.21.15.7:0/3253558092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0102090 msgr2=0x7fb5a0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/3253558092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0102090 0x7fb5a0102470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb590009900 tx=0x7fb59002bc50 comp rx=0 tx=0).stop 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 -- 172.21.15.7:0/3253558092 shutdown_connections 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/3253558092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0107160 0x7fb5a010ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/3253558092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a0102a40 0x7fb5a0106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/3253558092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0102090 0x7fb5a0102470 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 -- 172.21.15.7:0/3253558092 >> 172.21.15.7:0/3253558092 conn(0x7fb5a00fd950 msgr2=0x7fb5a00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 -- 172.21.15.7:0/3253558092 shutdown_connections 2024-04-20T21:35:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.912+0000 7fb5a5a00640 1 -- 172.21.15.7:0/3253558092 wait complete. 2024-04-20T21:35:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.913+0000 7fb5a5a00640 1 Processor -- start 2024-04-20T21:35:07.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 -- start start 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0107160 0x7fb5a01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a019a7c0 0x7fb5a019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0112220 con 0x7fb5a019a7c0 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01120a0 con 0x7fb5a0102090 2024-04-20T21:35:07.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb5a5a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a01123a0 con 0x7fb5a0107160 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb59fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb59fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40606/0 (socket says 172.21.15.7:40606) 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb59fe00640 1 -- 172.21.15.7:0/2355103027 learned_addr learned my addr 172.21.15.7:0/2355103027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.914+0000 7fb59f400640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0107160 0x7fb5a01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb5a5000640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a019a7c0 0x7fb5a019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 -- 172.21.15.7:0/2355103027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0107160 msgr2=0x7fb5a01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0107160 0x7fb5a01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 -- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a019a7c0 msgr2=0x7fb5a019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a019a7c0 0x7fb5a019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 -- 172.21.15.7:0/2355103027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a019b350 con 0x7fb5a0102090 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.915+0000 7fb59fe00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb590006010 tx=0x7fb59002cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.916+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590042020 con 0x7fb5a0102090 2024-04-20T21:35:07.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.916+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb590004450 con 0x7fb5a0102090 2024-04-20T21:35:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.916+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590003e30 con 0x7fb5a0102090 2024-04-20T21:35:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.916+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a019b5e0 con 0x7fb5a0102090 2024-04-20T21:35:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.916+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a01a77c0 con 0x7fb5a0102090 2024-04-20T21:35:07.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.917+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a0103560 con 0x7fb5a0102090 2024-04-20T21:35:07.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.919+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb590045020 con 0x7fb5a0102090 2024-04-20T21:35:07.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.921+0000 7fb59cc00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 0x7fb5840a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.921+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb5900f25a0 con 0x7fb5a0102090 2024-04-20T21:35:07.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.921+0000 7fb59f400640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 0x7fb5840a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:07.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.923+0000 7fb59f400640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 0x7fb5840a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5a010f180 tx=0x7fb58c005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:07.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:07.925+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb59004c020 con 0x7fb5a0102090 2024-04-20T21:35:07.977 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/types.h... yes 2024-04-20T21:35:08.089 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/stat.h... yes 2024-04-20T21:35:08.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.215+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5a0102a40 con 0x7fb5a0102090 2024-04-20T21:35:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.216+0000 7fb59cc00640 1 -- 172.21.15.7:0/2355103027 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7fb5900ba030 con 0x7fb5a0102090 2024-04-20T21:35:08.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:35:08.215 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:35:06.444983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24559,24575,24535]},"id":1}]} 2024-04-20T21:35:08.215 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-04-20T21:35:08.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 msgr2=0x7fb5840a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 0x7fb5840a61d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5a010f180 tx=0x7fb58c005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:08.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 msgr2=0x7fb5a01a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb590006010 tx=0x7fb59002cb50 comp rx=0 tx=0).stop 2024-04-20T21:35:08.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 shutdown_connections 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5a019a7c0 0x7fb5a019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb5a0107160 0x7fb5a01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb5840a3d10 0x7fb5840a61d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 --2- 172.21.15.7:0/2355103027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5a0102090 0x7fb5a01a0650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.218+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 >> 172.21.15.7:0/2355103027 conn(0x7fb5a00fd950 msgr2=0x7fb5a00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.219+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 shutdown_connections 2024-04-20T21:35:08.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.219+0000 7fb5a5a00640 1 -- 172.21.15.7:0/2355103027 wait complete. 2024-04-20T21:35:08.233 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdlib.h... yes 2024-04-20T21:35:08.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:35:08.368 INFO:tasks.workunit.client.0.smithi007.stdout:checking for string.h... yes 2024-04-20T21:35:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.424+0000 7f6861c00640 1 -- 172.21.15.7:0/787831092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076db0 msgr2=0x7f685c112890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.424+0000 7f6861c00640 1 --2- 172.21.15.7:0/787831092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076db0 0x7f685c112890 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6848009930 tx=0x7f684802bc30 comp rx=0 tx=0).stop 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.424+0000 7f6861c00640 1 -- 172.21.15.7:0/787831092 shutdown_connections 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.424+0000 7f6861c00640 1 --2- 172.21.15.7:0/787831092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076db0 0x7f685c112890 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.424+0000 7f6861c00640 1 --2- 172.21.15.7:0/787831092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 0x7f685c1151c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.425+0000 7f6861c00640 1 --2- 172.21.15.7:0/787831092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076490 0x7f685c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.425+0000 7f6861c00640 1 -- 172.21.15.7:0/787831092 >> 172.21.15.7:0/787831092 conn(0x7f685c0ffbe0 msgr2=0x7f685c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.425+0000 7f6861c00640 1 -- 172.21.15.7:0/787831092 shutdown_connections 2024-04-20T21:35:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.425+0000 7f6861c00640 1 -- 172.21.15.7:0/787831092 wait complete. 2024-04-20T21:35:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 Processor -- start 2024-04-20T21:35:08.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 -- start start 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076db0 0x7f685c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 0x7f685c1a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f685c118920 con 0x7f685c112dd0 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1187a0 con 0x7f685c076490 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.426+0000 7f6861c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f685c118aa0 con 0x7f685c076db0 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40630/0 (socket says 172.21.15.7:40630) 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 -- 172.21.15.7:0/3424622712 learned_addr learned my addr 172.21.15.7:0/3424622712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685a000640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076db0 0x7f685c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685b400640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 0x7f685c1a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 -- 172.21.15.7:0/3424622712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076db0 msgr2=0x7f685c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076db0 0x7f685c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 -- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 msgr2=0x7f685c1a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 0x7f685c1a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 -- 172.21.15.7:0/3424622712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c1a3e80 con 0x7f685c076490 2024-04-20T21:35:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.427+0000 7f685aa00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.430+0000 7f6853400640 1 -- 172.21.15.7:0/3424622712 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c00cd80 con 0x7f685c076490 2024-04-20T21:35:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.431+0000 7f6853400640 1 -- 172.21.15.7:0/3424622712 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c010910 con 0x7f685c076490 2024-04-20T21:35:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.431+0000 7f6853400640 1 -- 172.21.15.7:0/3424622712 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c018990 con 0x7f685c076490 2024-04-20T21:35:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.431+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c1a4170 con 0x7f685c076490 2024-04-20T21:35:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.431+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c1aba00 con 0x7f685c076490 2024-04-20T21:35:08.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.435+0000 7f6853400640 1 -- 172.21.15.7:0/3424622712 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f684c018b30 con 0x7f685c076490 2024-04-20T21:35:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 Processor -- start 2024-04-20T21:35:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 -- start start 2024-04-20T21:35:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5a8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5646cb505050 con 0x5646cb59ffb0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646cb4ab490 con 0x5646cb5a94b0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.436+0000 7f686499a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5646cb501010 con 0x5646cb5a08d0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53446/0 (socket says 172.21.15.7:53446) 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685b400640 1 -- 172.21.15.7:0/1018712447 learned_addr learned my addr 172.21.15.7:0/1018712447 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685aa00640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5a8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f6853400640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 0x7f68400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f6853400640 1 -- 172.21.15.7:0/3424622712 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f684c0d0ad0 con 0x7f685c076490 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 -- 172.21.15.7:0/1018712447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 msgr2=0x5646cb5a8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5a8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 -- 172.21.15.7:0/1018712447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 msgr2=0x5646cb5a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685a000640 1 -- 172.21.15.7:0/1018712447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cb4b3070 con 0x5646cb5a94b0 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685b400640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.437+0000 7f685b400640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 0x7f68400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.438+0000 7f685a000640 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ab8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6848009230 tx=0x7f6848009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4662d1fa6a0530b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.438+0000 7f685b400640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 0x7f68400a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6854003d50 tx=0x7f6854004a40 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.438+0000 7f6847e00640 1 -- 172.21.15.7:0/1018712447 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848042070 con 0x5646cb5a94b0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.438+0000 7f6847e00640 1 -- 172.21.15.7:0/1018712447 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684802c690 con 0x5646cb5a94b0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.438+0000 7f6847e00640 1 -- 172.21.15.7:0/1018712447 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848004b60 con 0x5646cb5a94b0 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 -- 172.21.15.7:0/1018712447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 msgr2=0x5646cb5ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ab8a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6848009230 tx=0x7f6848009610 comp rx=0 tx=0).stop 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 -- 172.21.15.7:0/1018712447 shutdown_connections 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ab8a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5a8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 --2- 172.21.15.7:0/1018712447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 -- 172.21.15.7:0/1018712447 >> 172.21.15.7:0/1018712447 conn(0x5646cb59a6e0 msgr2=0x5646cb59aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 -- 172.21.15.7:0/1018712447 shutdown_connections 2024-04-20T21:35:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.439+0000 7f686499a740 1 -- 172.21.15.7:0/1018712447 wait complete. 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.440+0000 7f686499a740 1 -- start start 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.440+0000 7f686499a740 1 Processor -- start 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 0x5646cb5a6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5646cb5ac240 con 0x5646cb5a08d0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646cb4be860 con 0x5646cb59ffb0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5646cb5ac3c0 con 0x5646cb5a94b0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f686499a740 10 client.0 _ll_register_callbacks cb 0x5646cb48cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 0x5646cb5a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44432/0 (socket says 172.21.15.7:44432) 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 -- 172.21.15.7:0/1952014176 learned_addr learned my addr 172.21.15.7:0/1952014176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 -- 172.21.15.7:0/1952014176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 msgr2=0x5646cb5a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 0x5646cb5a6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 -- 172.21.15.7:0/1952014176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 msgr2=0x5646cb5a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685b400640 1 -- 172.21.15.7:0/1952014176 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cb4c2950 con 0x5646cb5a94b0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.441+0000 7f685aa00640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 0x5646cb5a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f685b400640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6854020a20 tx=0x7f6854004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7798aa99b1a9111 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f6844c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854010bb0 con 0x5646cb5a94b0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646cb512a70 con 0x5646cb5a94b0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6854010d50 con 0x5646cb5a94b0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.442+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6854030050 con 0x5646cb5a94b0 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 10 client.25708 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646cb490820 con 0x5646cb5a94b0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5646cb512a70 con 0x5646cb5a94b0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f685401b6a0 con 0x5646cb5a94b0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 10 client.25708 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 10 client.25708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 20 client.25708 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb5b9490 0x5646cb5bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 4 client.25708 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646cb50c8e0 con 0x5646cb5b9490 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.443+0000 7f685aa00640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb5b9490 0x5646cb5bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.444+0000 7f685aa00640 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb5b9490 0x5646cb5bb8c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37780bea76cf7c6 server_cookie=773f6d4f6d470a0b in_seq=0 out_seq=0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.444+0000 7f6844c00640 10 client.25708 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f6844c00640 1 -- 172.21.15.7:0/1952014176 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646cb50c8e0 con 0x5646cb5b9490 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f6844c00640 10 client.25708 handle_command_reply: tid=0 2024-04-20T21:35:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 client.25708 shutdown 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 msgr2=0x5646cb5a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6854020a20 tx=0x7f6854004ab0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 shutdown_connections 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a94b0 0x5646cb5a6fa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb5b9490 0x5646cb5bb8c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 --2- 172.21.15.7:0/1952014176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb59ffb0 0x5646cb5a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 >> 172.21.15.7:0/1952014176 conn(0x5646cb59a6e0 msgr2=0x5646cb59c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f6844c00640 0 client.25708 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 shutdown_connections 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 1 -- 172.21.15.7:0/1952014176 wait complete. 2024-04-20T21:35:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.445+0000 7f686499a740 20 client.25708 trim_cache size 0 max 16384 2024-04-20T21:35:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.460+0000 7f686499a740 1 Processor -- start 2024-04-20T21:35:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 -- start start 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b6a50 con 0x5646cb5a08d0 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b4310 con 0x5646cb5a94b0 2024-04-20T21:35:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f686499a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b2160 con 0x5646cb59ffb0 2024-04-20T21:35:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44440/0 (socket says 172.21.15.7:44440) 2024-04-20T21:35:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685aa00640 1 -- 172.21.15.7:0/1781765196 learned_addr learned my addr 172.21.15.7:0/1781765196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.461+0000 7f685a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53464/0 (socket says 172.21.15.7:53464) 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 -- 172.21.15.7:0/1781765196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 msgr2=0x5646cb5995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 -- 172.21.15.7:0/1781765196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 msgr2=0x5646cb5b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 -- 172.21.15.7:0/1781765196 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cb5a0700 con 0x5646cb59ffb0 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685b400640 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f685aa00640 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ce270 tx=0x7f684c0d2f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=623b8ee5df586bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f6847400640 1 -- 172.21.15.7:0/1781765196 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c0a56d0 con 0x5646cb59ffb0 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f6847400640 1 -- 172.21.15.7:0/1781765196 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c09ae10 con 0x5646cb59ffb0 2024-04-20T21:35:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.462+0000 7f6847400640 1 -- 172.21.15.7:0/1781765196 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c09c1c0 con 0x5646cb59ffb0 2024-04-20T21:35:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 -- 172.21.15.7:0/1781765196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 msgr2=0x5646cb5a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f684c0ce270 tx=0x7f684c0d2f90 comp rx=0 tx=0).stop 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 -- 172.21.15.7:0/1781765196 shutdown_connections 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb5a08d0 0x5646cb5b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 --2- 172.21.15.7:0/1781765196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb59ffb0 0x5646cb5a8c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 -- 172.21.15.7:0/1781765196 >> 172.21.15.7:0/1781765196 conn(0x5646cb59a6e0 msgr2=0x5646cb598eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.463+0000 7f686499a740 1 -- 172.21.15.7:0/1781765196 shutdown_connections 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.464+0000 7f686499a740 1 -- 172.21.15.7:0/1781765196 wait complete. 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.464+0000 7f686499a740 1 -- start start 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.464+0000 7f686499a740 1 Processor -- start 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b27d0 con 0x5646cb59ffb0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b3c40 con 0x5646cb5a94b0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1b2ea0 con 0x5646cb5a08d0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f686499a740 10 client.0 _ll_register_callbacks cb 0x7f6854020830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40660/0 (socket says 172.21.15.7:40660) 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685a000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 -- 172.21.15.7:0/3196026652 learned_addr learned my addr 172.21.15.7:0/3196026652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 -- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 msgr2=0x5646cb5ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 -- 172.21.15.7:0/3196026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 msgr2=0x5646cb5acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.465+0000 7f685b400640 1 -- 172.21.15.7:0/3196026652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cb2173f0 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f685b400640 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6854004940 tx=0x7f6854005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09fdc2b5be2a970 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f6844c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685401b850 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6854010800 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684c09ae10 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6854010a20 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f686499a740 10 client.25718 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646cb2e17d0 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.466+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684c09ae10 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f685401dc60 con 0x5646cb5a94b0 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 10 client.25718 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 10 client.25718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 20 client.25718 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:08.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb61ff90 0x5646cb6223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 4 client.25718 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:08.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646cb4885a0 con 0x5646cb61ff90 2024-04-20T21:35:08.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f685aa00640 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb61ff90 0x5646cb6223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f685aa00640 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb61ff90 0x5646cb6223b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d755bc669e34567f server_cookie=999524f0b70669bf in_seq=0 out_seq=0 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.467+0000 7f6844c00640 10 client.25718 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f6844c00640 1 -- 172.21.15.7:0/3196026652 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646cb4885a0 con 0x5646cb61ff90 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f6844c00640 10 client.25718 handle_command_reply: tid=0 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 client.25718 shutdown 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 msgr2=0x5646cb5ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6854004940 tx=0x7f6854005ea0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 shutdown_connections 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5646cb61ff90 0x5646cb6223b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646cb5a94b0 0x5646cb5ada10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5646cb5a08d0 0x5646cb5ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 --2- 172.21.15.7:0/3196026652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5646cb59ffb0 0x5646cb5acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f6844c00640 0 client.25718 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 >> 172.21.15.7:0/3196026652 conn(0x5646cb59a6e0 msgr2=0x5646cb59c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 shutdown_connections 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 1 -- 172.21.15.7:0/3196026652 wait complete. 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.468+0000 7f686499a740 20 client.25718 trim_cache size 0 max 16384 2024-04-20T21:35:08.471 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 msgr2=0x7f68400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 0x7f68400a6260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6854003d50 tx=0x7f6854004a40 comp rx=0 tx=0).stop 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 msgr2=0x7f685c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 shutdown_connections 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f685c112dd0 0x7f685c1a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f685c076db0 0x7f685c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f68400a3da0 0x7f68400a6260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 --2- 172.21.15.7:0/3424622712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c076490 0x7f685c19eee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 >> 172.21.15.7:0/3424622712 conn(0x7f685c0ffbe0 msgr2=0x7f685c114560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 shutdown_connections 2024-04-20T21:35:08.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.469+0000 7f6861c00640 1 -- 172.21.15.7:0/3424622712 wait complete. 2024-04-20T21:35:08.485 INFO:tasks.workunit.client.0.smithi007.stdout:checking for memory.h... yes 2024-04-20T21:35:08.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:35:08.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:08 smithi007 ceph-mon[16867]: pgmap v56: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-04-20T21:35:08.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:08 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:35:08.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:08 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2355103027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:08.580 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strings.h... yes 2024-04-20T21:35:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 -- 172.21.15.7:0/313944114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 msgr2=0x7f3f80108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 --2- 172.21.15.7:0/313944114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f80108ac0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f68009900 tx=0x7f3f6802bc30 comp rx=0 tx=0).stop 2024-04-20T21:35:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 -- 172.21.15.7:0/313944114 shutdown_connections 2024-04-20T21:35:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 --2- 172.21.15.7:0/313944114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f80108ac0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 --2- 172.21.15.7:0/313944114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f80115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 --2- 172.21.15.7:0/313944114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 0x7f3f8010a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 -- 172.21.15.7:0/313944114 >> 172.21.15.7:0/313944114 conn(0x7f3f80077dd0 msgr2=0x7f3f80106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 -- 172.21.15.7:0/313944114 shutdown_connections 2024-04-20T21:35:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.675+0000 7f3f85200640 1 -- 172.21.15.7:0/313944114 wait complete. 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.676+0000 7f3f85200640 1 Processor -- start 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 -- start start 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f8019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 0x7f3f801a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80118a20 con 0x7f3f8010a400 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801188a0 con 0x7f3f80108680 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f85200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80118ba0 con 0x7f3f80109000 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40680/0 (socket says 172.21.15.7:40680) 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f7de00640 1 -- 172.21.15.7:0/1930574055 learned_addr learned my addr 172.21.15.7:0/1930574055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f75400640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f8019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.677+0000 7f3f7e800640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 0x7f3f801a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 -- 172.21.15.7:0/1930574055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 msgr2=0x7f3f8019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f8019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 -- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 msgr2=0x7f3f801a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 0x7f3f801a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 -- 172.21.15.7:0/1930574055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f801a3ef0 con 0x7f3f80108680 2024-04-20T21:35:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f75400640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f8019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f7de00640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000d950 tx=0x7f3f7000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f77400640 1 -- 172.21.15.7:0/1930574055 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70017030 con 0x7f3f80108680 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f77400640 1 -- 172.21.15.7:0/1930574055 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70004020 con 0x7f3f80108680 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f77400640 1 -- 172.21.15.7:0/1930574055 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700047a0 con 0x7f3f80108680 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f801a41e0 con 0x7f3f80108680 2024-04-20T21:35:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.678+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801aba70 con 0x7f3f80108680 2024-04-20T21:35:08.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f77400640 1 -- 172.21.15.7:0/1930574055 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3f700041d0 con 0x7f3f80108680 2024-04-20T21:35:08.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 Processor -- start 2024-04-20T21:35:08.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 -- start start 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a593390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a59bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e86a4f8050 con 0x55e86a592fb0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e86a49e490 con 0x55e86a59c4b0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.682+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e86a4f4010 con 0x55e86a5938d0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40690/0 (socket says 172.21.15.7:40690) 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 -- 172.21.15.7:0/1817598926 learned_addr learned my addr 172.21.15.7:0/1817598926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f7e800640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a593390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f7de00640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a59bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 -- 172.21.15.7:0/1817598926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 msgr2=0x55e86a59bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a59bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 -- 172.21.15.7:0/1817598926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 msgr2=0x55e86a593390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a593390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f75400640 1 -- 172.21.15.7:0/1817598926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86a4a6070 con 0x55e86a59c4b0 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.683+0000 7f3f7e800640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a593390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f77400640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 0x7f3f640a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f75400640 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f6802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a24727b557109 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f77400640 1 -- 172.21.15.7:0/1930574055 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3f700d0160 con 0x7f3f80108680 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f6f400640 1 -- 172.21.15.7:0/1817598926 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68042070 con 0x55e86a59c4b0 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f6f400640 1 -- 172.21.15.7:0/1817598926 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f68004240 con 0x55e86a59c4b0 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f6f400640 1 -- 172.21.15.7:0/1817598926 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680046c0 con 0x55e86a59c4b0 2024-04-20T21:35:08.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.684+0000 7f3f7e800640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 0x7f3f640a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1817598926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 msgr2=0x55e86a59e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3f68000c00 tx=0x7f3f6802c5d0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f7e800640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 0x7f3f640a6260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f78005fd0 tx=0x7f3f78009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1817598926 shutdown_connections 2024-04-20T21:35:08.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a59e8a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a59bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1817598926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a593390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1817598926 >> 172.21.15.7:0/1817598926 conn(0x55e86a58d6e0 msgr2=0x55e86a58dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1817598926 shutdown_connections 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.685+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1817598926 wait complete. 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 -- start start 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 Processor -- start 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a599a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a599fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e86a59f240 con 0x55e86a5938d0 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e86a4b1860 con 0x55e86a592fb0 2024-04-20T21:35:08.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.686+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e86a59f3c0 con 0x55e86a59c4b0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f87fa2740 10 client.0 _ll_register_callbacks cb 0x55e86a47ffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40700/0 (socket says 172.21.15.7:40700) 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 -- 172.21.15.7:0/3318985667 learned_addr learned my addr 172.21.15.7:0/3318985667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f75400640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a599a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7e800640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a599fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 -- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 msgr2=0x55e86a599fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a599fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 -- 172.21.15.7:0/3318985667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 msgr2=0x55e86a599a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a599a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 -- 172.21.15.7:0/3318985667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86a4b5950 con 0x55e86a592fb0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f7de00640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cd960 tx=0x7f3f70005130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad1991e81f24212 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.687+0000 7f3f63e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7009bf20 con 0x55e86a592fb0 2024-04-20T21:35:08.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3f700a0150 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f68004240 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f700d3e60 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 10 client.25730 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e86a483820 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f68004240 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f3f700a4930 con 0x55e86a592fb0 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 10 client.25730 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 10 client.25730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 20 client.25730 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a5ac380 0x55e86a5ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 4 client.25730 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.688+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e86a505a70 con 0x55e86a5ac380 2024-04-20T21:35:08.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.689+0000 7f3f75400640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a5ac380 0x55e86a5ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.692 INFO:tasks.workunit.client.0.smithi007.stdout:checking for inttypes.h... yes 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.689+0000 7f3f75400640 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a5ac380 0x55e86a5ae770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf405095d1d77270 server_cookie=476bd34d524ccb1d in_seq=0 out_seq=0 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.689+0000 7f3f63e00640 10 client.25730 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f63e00640 1 -- 172.21.15.7:0/3318985667 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e86a505a70 con 0x55e86a5ac380 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f63e00640 10 client.25730 handle_command_reply: tid=0 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 client.25730 shutdown 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 msgr2=0x55e86a599520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cd960 tx=0x7f3f70005130 comp rx=0 tx=0).stop 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 shutdown_connections 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a5ac380 0x55e86a5ae770 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a599fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a599a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f63e00640 0 client.25730 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/3318985667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a599520 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.690+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 >> 172.21.15.7:0/3318985667 conn(0x55e86a58d6e0 msgr2=0x55e86a58f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.691+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 shutdown_connections 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.691+0000 7f3f87fa2740 1 -- 172.21.15.7:0/3318985667 wait complete. 2024-04-20T21:35:08.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.691+0000 7f3f87fa2740 20 client.25730 trim_cache size 0 max 16384 2024-04-20T21:35:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:08 smithi016 ceph-mon[21409]: pgmap v56: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-04-20T21:35:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:08 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:35:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:08 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2355103027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.707+0000 7f3f87fa2740 1 Processor -- start 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.707+0000 7f3f87fa2740 1 -- start start 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a59bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a5b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b6ac0 con 0x55e86a592fb0 2024-04-20T21:35:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b4380 con 0x55e86a59c4b0 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b21d0 con 0x55e86a5938d0 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f75400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f7e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a5b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f75400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44486/0 (socket says 172.21.15.7:44486) 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f75400640 1 -- 172.21.15.7:0/1542277746 learned_addr learned my addr 172.21.15.7:0/1542277746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.708+0000 7f3f7de00640 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a59bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 -- 172.21.15.7:0/1542277746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 msgr2=0x55e86a5b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a5b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 -- 172.21.15.7:0/1542277746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 msgr2=0x55e86a59bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a59bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 -- 172.21.15.7:0/1542277746 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86a20a3f0 con 0x55e86a5938d0 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f75400640 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802c240 tx=0x7f3f68005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e621f9c3e978cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f6ea00640 1 -- 172.21.15.7:0/1542277746 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f68042040 con 0x55e86a5938d0 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f6ea00640 1 -- 172.21.15.7:0/1542277746 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f680035f0 con 0x55e86a5938d0 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.709+0000 7f3f6ea00640 1 -- 172.21.15.7:0/1542277746 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6802cd70 con 0x55e86a5938d0 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1542277746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 msgr2=0x55e86a5a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3f6802c240 tx=0x7f3f68005410 comp rx=0 tx=0).stop 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1542277746 shutdown_connections 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a59c4b0 0x55e86a5b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a5938d0 0x55e86a5a8100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/1542277746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a592fb0 0x55e86a59bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1542277746 >> 172.21.15.7:0/1542277746 conn(0x55e86a58d6e0 msgr2=0x55e86a598680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.710+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1542277746 shutdown_connections 2024-04-20T21:35:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.711+0000 7f3f87fa2740 1 -- 172.21.15.7:0/1542277746 wait complete. 2024-04-20T21:35:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.711+0000 7f3f87fa2740 1 -- start start 2024-04-20T21:35:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.711+0000 7f3f87fa2740 1 Processor -- start 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a5a4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a59f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b2840 con 0x55e86a5938d0 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b3cb0 con 0x55e86a592fb0 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b2f10 con 0x55e86a59c4b0 2024-04-20T21:35:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f87fa2740 10 client.0 _ll_register_callbacks cb 0x7f3f700d39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40710/0 (socket says 172.21.15.7:40710) 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 -- 172.21.15.7:0/707201925 learned_addr learned my addr 172.21.15.7:0/707201925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7e800640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a59f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f75400640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a5a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 -- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 msgr2=0x55e86a59f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a59f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 -- 172.21.15.7:0/707201925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 msgr2=0x55e86a5a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a5a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.712+0000 7f3f7de00640 1 -- 172.21.15.7:0/707201925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86a2d47d0 con 0x55e86a592fb0 2024-04-20T21:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f7de00640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f70098b20 tx=0x7f3f700a4b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8282e950b73df42f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f63e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70014070 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3f70097ea0 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f680035f0 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f7009bab0 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f87fa2740 10 client.25736 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e86a612150 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.713+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f680035f0 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f3f700980a0 con 0x55e86a592fb0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 10 client.25736 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 10 client.25736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 20 client.25736 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a612890 0x55e86a614cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 4 client.25736 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e86a4ff8e0 con 0x55e86a612890 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f75400640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a612890 0x55e86a614cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f75400640 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a612890 0x55e86a614cb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ff969013ed63c83 server_cookie=1fcf8d0be72c4bc0 in_seq=0 out_seq=0 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.714+0000 7f3f63e00640 10 client.25736 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f63e00640 1 -- 172.21.15.7:0/707201925 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e86a4ff8e0 con 0x55e86a612890 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f63e00640 10 client.25736 handle_command_reply: tid=0 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 client.25736 shutdown 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 msgr2=0x55e86a5a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f70098b20 tx=0x7f3f700a4b40 comp rx=0 tx=0).stop 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 shutdown_connections 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e86a612890 0x55e86a614cb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e86a59c4b0 0x55e86a59f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e86a5938d0 0x55e86a5a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 --2- 172.21.15.7:0/707201925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e86a592fb0 0x55e86a5a4320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 >> 172.21.15.7:0/707201925 conn(0x55e86a58d6e0 msgr2=0x55e86a58beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f63e00640 0 client.25736 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 shutdown_connections 2024-04-20T21:35:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 1 -- 172.21.15.7:0/707201925 wait complete. 2024-04-20T21:35:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.715+0000 7f3f87fa2740 20 client.25736 trim_cache size 0 max 16384 2024-04-20T21:35:08.718 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 msgr2=0x7f3f640a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 0x7f3f640a6260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f78005fd0 tx=0x7f3f78009500 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 msgr2=0x7f3f8019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000d950 tx=0x7f3f7000de10 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 shutdown_connections 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f8010a400 0x7f3f801a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f80109000 0x7f3f8019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3f640a3da0 0x7f3f640a6260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 --2- 172.21.15.7:0/1930574055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f80108680 0x7f3f8019ef00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 >> 172.21.15.7:0/1930574055 conn(0x7f3f80077dd0 msgr2=0x7f3f80078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 shutdown_connections 2024-04-20T21:35:08.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.716+0000 7f3f85200640 1 -- 172.21.15.7:0/1930574055 wait complete. 2024-04-20T21:35:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:08 smithi186 ceph-mon[22157]: pgmap v56: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 177 KiB/s wr, 30 op/s 2024-04-20T21:35:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:08 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:35:08.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:08 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2355103027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:08.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:35:08.790 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdint.h... yes 2024-04-20T21:35:08.907 INFO:tasks.workunit.client.0.smithi007.stdout:checking for unistd.h... yes 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 -- 172.21.15.7:0/1217095071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe010c510 msgr2=0x7f5fe010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/1217095071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe010c510 0x7f5fe010e9b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009ca0 tx=0x7f5fd802c190 comp rx=0 tx=0).stop 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 -- 172.21.15.7:0/1217095071 shutdown_connections 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/1217095071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe010c510 0x7f5fe010e9b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/1217095071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe0103af0 0x7f5fe010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/1217095071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe01031d0 0x7f5fe01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 -- 172.21.15.7:0/1217095071 >> 172.21.15.7:0/1217095071 conn(0x7f5fe0078010 msgr2=0x7f5fe0078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 -- 172.21.15.7:0/1217095071 shutdown_connections 2024-04-20T21:35:08.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.973+0000 7f5fe6a00640 1 -- 172.21.15.7:0/1217095071 wait complete. 2024-04-20T21:35:08.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.974+0000 7f5fe6a00640 1 Processor -- start 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.974+0000 7f5fe6a00640 1 -- start start 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.974+0000 7f5fe6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 0x7f5fe019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.974+0000 7f5fe6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 0x7f5fe019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.974+0000 7f5fe6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 0x7f5fe01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fe6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01122d0 con 0x7f5fe010c510 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fe6a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0112150 con 0x7f5fe01031d0 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fe6a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0112450 con 0x7f5fe0103af0 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 0x7f5fe019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 0x7f5fe019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 0x7f5fe019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44514/0 (socket says 172.21.15.7:44514) 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fd7400640 1 -- 172.21.15.7:0/962146827 learned_addr learned my addr 172.21.15.7:0/962146827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 0x7f5fe01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 -- 172.21.15.7:0/962146827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 msgr2=0x7f5fe019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 0x7f5fe019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 -- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 msgr2=0x7f5fe01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 0x7f5fe01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 -- 172.21.15.7:0/962146827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe0198d30 con 0x7f5fe01031d0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 0x7f5fe01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.975+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 0x7f5fe019e400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000e9f0 tx=0x7f5fd000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.976+0000 7f5fdd600640 1 -- 172.21.15.7:0/962146827 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000ccc0 con 0x7f5fe01031d0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.976+0000 7f5fdd600640 1 -- 172.21.15.7:0/962146827 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd0010910 con 0x7f5fe01031d0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.976+0000 7f5fdd600640 1 -- 172.21.15.7:0/962146827 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0018960 con 0x7f5fe01031d0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.976+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe0199020 con 0x7f5fe01031d0 2024-04-20T21:35:08.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.976+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe01a52c0 con 0x7f5fe01031d0 2024-04-20T21:35:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.979+0000 7f5fe9789740 1 Processor -- start 2024-04-20T21:35:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.979+0000 7f5fdd600640 1 -- 172.21.15.7:0/962146827 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5fd0010ac0 con 0x7f5fe01031d0 2024-04-20T21:35:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 -- start start 2024-04-20T21:35:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7bf17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7bfccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595a7b52050 con 0x5595a7bf13c0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595a7af8490 con 0x5595a7bf1ce0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe9789740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5595a7b4e010 con 0x5595a7bfa8c0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fe4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7bf17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40748/0 (socket says 172.21.15.7:40748) 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.980+0000 7f5fdfe00640 1 -- 172.21.15.7:0/759739201 learned_addr learned my addr 172.21.15.7:0/759739201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fd7400640 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7bfccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 -- 172.21.15.7:0/759739201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 msgr2=0x5595a7bfccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7bfccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 -- 172.21.15.7:0/759739201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 msgr2=0x5595a7bf17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7bf17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 -- 172.21.15.7:0/759739201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a7b00070 con 0x5595a7bf1ce0 2024-04-20T21:35:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00ce220 tx=0x7f5fd00d9660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6a094863698787 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.981+0000 7f5fdd600640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 0x7f5fc40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.982+0000 7f5fd5200640 1 -- 172.21.15.7:0/759739201 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00e0020 con 0x5595a7bf1ce0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.982+0000 7f5fd5200640 1 -- 172.21.15.7:0/759739201 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd00d5040 con 0x5595a7bf1ce0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.982+0000 7f5fd5200640 1 -- 172.21.15.7:0/759739201 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd009c900 con 0x5595a7bf1ce0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.982+0000 7f5fdd600640 1 -- 172.21.15.7:0/962146827 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5fd00d0ba0 con 0x7f5fe01031d0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.982+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 0x7f5fc40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- 172.21.15.7:0/759739201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 msgr2=0x5595a7bfa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00ce220 tx=0x7f5fd00d9660 comp rx=0 tx=0).stop 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 0x7f5fc40a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8006010 tx=0x7f5fd8009360 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- 172.21.15.7:0/759739201 shutdown_connections 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bfa380 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7bfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 --2- 172.21.15.7:0/759739201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7bf17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- 172.21.15.7:0/759739201 >> 172.21.15.7:0/759739201 conn(0x5595a7bebaf0 msgr2=0x5595a7bebed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- 172.21.15.7:0/759739201 shutdown_connections 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- 172.21.15.7:0/759739201 wait complete. 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.983+0000 7f5fe9789740 1 -- start start 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 Processor -- start 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5595a7bfd6b0 con 0x5595a7bf1ce0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5595a7b0b860 con 0x5595a7bfa8c0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5595a7bfd830 con 0x5595a7bf13c0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.984+0000 7f5fe9789740 10 client.0 _ll_register_callbacks cb 0x5595a7ad9fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fe4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fe4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40764/0 (socket says 172.21.15.7:40764) 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fe4a00640 1 -- 172.21.15.7:0/2576441124 learned_addr learned my addr 172.21.15.7:0/2576441124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fd7400640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 -- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 msgr2=0x5595a7bf7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 -- 172.21.15.7:0/2576441124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf1ce0 msgr2=0x5595a7bf78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 -- 172.21.15.7:0/2576441124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a7b0f950 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bfa2b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00ce220 tx=0x7f5fd000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4285e42abe76fe9e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.985+0000 7f5fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd004e930 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5fd00991a0 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd00d5040 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fd004eb00 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 10 client.25744 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595a7add820 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd00d5040 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f5fd00ce560 con 0x5595a7bf13c0 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 10 client.25744 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.986+0000 7f5fe9789740 20 client.25744 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.987+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c0eb60 0x5595a7c10f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.987+0000 7f5fe9789740 4 client.25744 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.987+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595a7b5fa70 con 0x5595a7c0eb60 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.987+0000 7f5fd7400640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c0eb60 0x5595a7c10f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.988+0000 7f5fd7400640 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c0eb60 0x5595a7c10f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170e159d5f223154 server_cookie=44429eab5274ab0b in_seq=0 out_seq=0 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.988+0000 7f5fcd600640 10 client.25744 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.988+0000 7f5fcd600640 1 -- 172.21.15.7:0/2576441124 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595a7b5fa70 con 0x5595a7c0eb60 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.988+0000 7f5fcd600640 10 client.25744 handle_command_reply: tid=0 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.988+0000 7f5fe9789740 1 client.25744 shutdown 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 msgr2=0x5595a7bfa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bfa2b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00ce220 tx=0x7f5fd000cf60 comp rx=0 tx=0).stop 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 shutdown_connections 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bfa8c0 0x5595a7bf7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c0eb60 0x5595a7c10f50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 --2- 172.21.15.7:0/2576441124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bfa2b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fcd600640 0 client.25744 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 >> 172.21.15.7:0/2576441124 conn(0x5595a7bebaf0 msgr2=0x5595a7bed840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 shutdown_connections 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 1 -- 172.21.15.7:0/2576441124 wait complete. 2024-04-20T21:35:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:08.989+0000 7f5fe9789740 20 client.25744 trim_cache size 0 max 16384 2024-04-20T21:35:09.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.006+0000 7f5fe9789740 1 Processor -- start 2024-04-20T21:35:09.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 -- start start 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bf6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 0x5595a7be65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01ae2f0 con 0x5595a7bfa8c0 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01abbb0 con 0x5595a7bf1ce0 2024-04-20T21:35:09.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe9789740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01a9a00 con 0x5595a7bf13c0 2024-04-20T21:35:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40776/0 (socket says 172.21.15.7:40776) 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fd7400640 1 -- 172.21.15.7:0/178072836 learned_addr learned my addr 172.21.15.7:0/178072836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 0x5595a7be65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.007+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bf6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fe4a00640 1 -- 172.21.15.7:0/178072836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 msgr2=0x5595a7bf6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bf6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fe4a00640 1 -- 172.21.15.7:0/178072836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 msgr2=0x5595a7bf9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fe4a00640 1 -- 172.21.15.7:0/178072836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a78643f0 con 0x5595a7bfa8c0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.008+0000 7f5fd7400640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.009+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 0x5595a7be65c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8037040 tx=0x7f5fd8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0b7add1b345f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.009+0000 7f5fcfe00640 1 -- 172.21.15.7:0/178072836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8039040 con 0x5595a7bfa8c0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.009+0000 7f5fcfe00640 1 -- 172.21.15.7:0/178072836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8032e50 con 0x5595a7bfa8c0 2024-04-20T21:35:09.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.009+0000 7f5fcfe00640 1 -- 172.21.15.7:0/178072836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8034a60 con 0x5595a7bfa8c0 2024-04-20T21:35:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 -- 172.21.15.7:0/178072836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 msgr2=0x5595a7be65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 0x5595a7be65c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8037040 tx=0x7f5fd8040000 comp rx=0 tx=0).stop 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 -- 172.21.15.7:0/178072836 shutdown_connections 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7bf9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bfa8c0 0x5595a7be65c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 --2- 172.21.15.7:0/178072836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bf13c0 0x5595a7bf6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 -- 172.21.15.7:0/178072836 >> 172.21.15.7:0/178072836 conn(0x5595a7bebaf0 msgr2=0x5595a7be5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.010+0000 7f5fe9789740 1 -- 172.21.15.7:0/178072836 shutdown_connections 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.011+0000 7f5fe9789740 1 -- 172.21.15.7:0/178072836 wait complete. 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.011+0000 7f5fe9789740 1 -- start start 2024-04-20T21:35:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.011+0000 7f5fe9789740 1 Processor -- start 2024-04-20T21:35:09.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7c0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7c0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01aa070 con 0x5595a7bf13c0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01ab4e0 con 0x5595a7bf1ce0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01aa740 con 0x5595a7bfa8c0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe9789740 10 client.0 _ll_register_callbacks cb 0x7f5fd00d4b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fdfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7c0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40778/0 (socket says 172.21.15.7:40778) 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fd7400640 1 -- 172.21.15.7:0/3907379600 learned_addr learned my addr 172.21.15.7:0/3907379600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.012+0000 7f5fe4a00640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7c0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 -- 172.21.15.7:0/3907379600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 msgr2=0x5595a7c0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7c0bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 -- 172.21.15.7:0/3907379600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 msgr2=0x5595a7c0b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7c0b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 -- 172.21.15.7:0/3907379600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a792e7d0 con 0x5595a7bf1ce0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fd7400640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009620 tx=0x7f5fc80043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474aee1df3837e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.013+0000 7f5fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.014+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8024030 con 0x5595a7bf1ce0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.014+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5fc8005020 con 0x5595a7bf1ce0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.014+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd8032e50 con 0x5595a7bf1ce0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.015+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fc8005220 con 0x5595a7bf1ce0 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.015+0000 7f5fe9789740 10 client.25754 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.015+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5595a7c740d0 con 0x5595a7bf1ce0 2024-04-20T21:35:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.016+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8032e50 con 0x5595a7bf1ce0 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.016+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f5fc8004960 con 0x5595a7bf1ce0 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 10 client.25754 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 20 client.25754 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c74860 0x5595a7c76c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 4 client.25754 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.017+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595a7b598e0 con 0x5595a7c74860 2024-04-20T21:35:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.018+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c74860 0x5595a7c76c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fdfe00640 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c74860 0x5595a7c76c80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45cbb2b969929dd server_cookie=6053a71026ad25ad in_seq=0 out_seq=0 2024-04-20T21:35:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fcd600640 10 client.25754 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fcd600640 1 -- 172.21.15.7:0/3907379600 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595a7b598e0 con 0x5595a7c74860 2024-04-20T21:35:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fcd600640 10 client.25754 handle_command_reply: tid=0 2024-04-20T21:35:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fe9789740 1 client.25754 shutdown 2024-04-20T21:35:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 msgr2=0x5595a7c0b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.019+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009620 tx=0x7f5fc80043c0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 shutdown_connections 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5595a7bf1ce0 0x5595a7c0b880 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5595a7c74860 0x5595a7c76c80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5595a7bfa8c0 0x5595a7c0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 --2- 172.21.15.7:0/3907379600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5595a7bf13c0 0x5595a7c0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fcd600640 0 client.25754 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.020+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 >> 172.21.15.7:0/3907379600 conn(0x5595a7bebaf0 msgr2=0x5595a7be6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.022+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 shutdown_connections 2024-04-20T21:35:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.022+0000 7f5fe9789740 1 -- 172.21.15.7:0/3907379600 wait complete. 2024-04-20T21:35:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.023+0000 7f5fe9789740 20 client.25754 trim_cache size 0 max 16384 2024-04-20T21:35:09.022 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 msgr2=0x7f5fc40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 0x7f5fc40a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8006010 tx=0x7f5fd8009360 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 msgr2=0x7f5fe019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 0x7f5fe019e400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000e9f0 tx=0x7f5fd000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 shutdown_connections 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fe010c510 0x7f5fe01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5fe0103af0 0x7f5fe019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5fc40a3da0 0x7f5fc40a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 --2- 172.21.15.7:0/962146827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fe01031d0 0x7f5fe019e400 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000e9f0 tx=0x7f5fd000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 >> 172.21.15.7:0/962146827 conn(0x7f5fe0078010 msgr2=0x7f5fe00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 shutdown_connections 2024-04-20T21:35:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:09.025+0000 7f5fe6a00640 1 -- 172.21.15.7:0/962146827 wait complete. 2024-04-20T21:35:09.048 INFO:tasks.workunit.client.0.smithi007.stdout:checking pthread.h usability... yes 2024-04-20T21:35:09.095 INFO:tasks.workunit.client.0.smithi007.stdout:checking pthread.h presence... yes 2024-04-20T21:35:09.096 INFO:tasks.workunit.client.0.smithi007.stdout:checking for pthread.h... yes 2024-04-20T21:35:09.219 INFO:tasks.workunit.client.0.smithi007.stdout:checking fcntl.h usability... yes 2024-04-20T21:35:09.278 INFO:tasks.workunit.client.0.smithi007.stdout:checking fcntl.h presence... yes 2024-04-20T21:35:09.280 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fcntl.h... yes 2024-04-20T21:35:09.406 INFO:tasks.workunit.client.0.smithi007.stdout:checking limits.h usability... yes 2024-04-20T21:35:09.464 INFO:tasks.workunit.client.0.smithi007.stdout:checking limits.h presence... yes 2024-04-20T21:35:09.466 INFO:tasks.workunit.client.0.smithi007.stdout:checking for limits.h... yes 2024-04-20T21:35:09.500 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdint.h... (cached) yes 2024-04-20T21:35:09.615 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h usability... yes 2024-04-20T21:35:09.676 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h presence... yes 2024-04-20T21:35:09.679 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/time.h... yes 2024-04-20T21:35:09.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:09 smithi016 ceph-mon[21409]: pgmap v57: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:09.714 INFO:tasks.workunit.client.0.smithi007.stdout:checking for unistd.h... (cached) yes 2024-04-20T21:35:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:09 smithi186 ceph-mon[22157]: pgmap v57: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:09.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:09 smithi007 ceph-mon[16867]: pgmap v57: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:09.846 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/vfs.h usability... yes 2024-04-20T21:35:09.909 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/vfs.h presence... yes 2024-04-20T21:35:09.912 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/vfs.h... yes 2024-04-20T21:35:10.031 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/limits.h usability... no 2024-04-20T21:35:10.043 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:35:10.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:35:10.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:35:10.089 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/limits.h presence... no 2024-04-20T21:35:10.091 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/limits.h... no 2024-04-20T21:35:10.169 INFO:tasks.workunit.client.0.smithi007.stdout:checking for an ANSI C-conforming const... yes 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 -- 172.21.15.7:0/86568126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 msgr2=0x7f40e8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 --2- 172.21.15.7:0/86568126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8102570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009900 tx=0x7f40dc02bc30 comp rx=0 tx=0).stop 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 -- 172.21.15.7:0/86568126 shutdown_connections 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 --2- 172.21.15.7:0/86568126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e8107260 0x7f40e810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 --2- 172.21.15.7:0/86568126 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8102b40 0x7f40e8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 --2- 172.21.15.7:0/86568126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8102570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 -- 172.21.15.7:0/86568126 >> 172.21.15.7:0/86568126 conn(0x7f40e80fda10 msgr2=0x7f40e80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 -- 172.21.15.7:0/86568126 shutdown_connections 2024-04-20T21:35:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.244+0000 7f40f0600640 1 -- 172.21.15.7:0/86568126 wait complete. 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.245+0000 7f40f0600640 1 Processor -- start 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.245+0000 7f40f0600640 1 -- start start 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.245+0000 7f40f0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.245+0000 7f40f0600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.245+0000 7f40f0600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e819af50 0x7f40e819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40f0600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8110170 con 0x7f40e819af50 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40f0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40e810fff0 con 0x7f40e8102190 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40f0600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81102f0 con 0x7f40e8107260 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40ed200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e819af50 0x7f40e819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40ed200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44568/0 (socket says 172.21.15.7:44568) 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40ed200640 1 -- 172.21.15.7:0/2209447569 learned_addr learned my addr 172.21.15.7:0/2209447569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 -- 172.21.15.7:0/2209447569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 msgr2=0x7f40e8196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 -- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e819af50 msgr2=0x7f40e819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e819af50 0x7f40e819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40edc00640 1 -- 172.21.15.7:0/2209447569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e819ba40 con 0x7f40e8102190 2024-04-20T21:35:10.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.246+0000 7f40ed200640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40edc00640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8196690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009850 tx=0x7f40dc0042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40e6000640 1 -- 172.21.15.7:0/2209447569 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc043020 con 0x7f40e8102190 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40e6000640 1 -- 172.21.15.7:0/2209447569 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc0044d0 con 0x7f40e8102190 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40e6000640 1 -- 172.21.15.7:0/2209447569 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc003b60 con 0x7f40e8102190 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e819bcd0 con 0x7f40e8102190 2024-04-20T21:35:10.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.247+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81a3440 con 0x7f40e8102190 2024-04-20T21:35:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.250+0000 7f40f337e740 1 Processor -- start 2024-04-20T21:35:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.250+0000 7f40e6000640 1 -- 172.21.15.7:0/2209447569 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f40dc046020 con 0x7f40e8102190 2024-04-20T21:35:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 -- start start 2024-04-20T21:35:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f9802800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5632f975bd20 con 0x5632f97f6f10 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632f97023e0 con 0x5632f9800410 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40f337e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5632f9757ce0 con 0x5632f97f7830 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40ed200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f9802800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40edc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53622/0 (socket says 172.21.15.7:53622) 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.251+0000 7f40ee600640 1 -- 172.21.15.7:0/2701992151 learned_addr learned my addr 172.21.15.7:0/2701992151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 -- 172.21.15.7:0/2701992151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 msgr2=0x5632f97ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 -- 172.21.15.7:0/2701992151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 msgr2=0x5632f97f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 -- 172.21.15.7:0/2701992151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f9721980 con 0x5632f9800410 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40e6000640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 0x7f40d00aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40ed200640 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f9802800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f40e80fdfa0 tx=0x7f40d8011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81a7c79a55de7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.252+0000 7f40d6a00640 1 -- 172.21.15.7:0/2701992151 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d801e070 con 0x5632f9800410 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40e6000640 1 -- 172.21.15.7:0/2209447569 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40dc0f26b0 con 0x7f40e8102190 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40d6a00640 1 -- 172.21.15.7:0/2701992151 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d800fa20 con 0x5632f9800410 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40d6a00640 1 -- 172.21.15.7:0/2701992151 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8019bf0 con 0x5632f9800410 2024-04-20T21:35:10.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40ee600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 0x7f40d00aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.254 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-20T21:35:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40f337e740 1 -- 172.21.15.7:0/2701992151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 msgr2=0x5632f9802800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40f337e740 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f9802800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f40e80fdfa0 tx=0x7f40d8011950 comp rx=0 tx=0).stop 2024-04-20T21:35:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.253+0000 7f40ee600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 0x7f40d00aa6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005f40 tx=0x7f40e00096a0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 -- 172.21.15.7:0/2701992151 shutdown_connections 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f9802800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 --2- 172.21.15.7:0/2701992151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 -- 172.21.15.7:0/2701992151 >> 172.21.15.7:0/2701992151 conn(0x5632f97f1640 msgr2=0x5632f97f1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 -- 172.21.15.7:0/2701992151 shutdown_connections 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 -- 172.21.15.7:0/2701992151 wait complete. 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 -- start start 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.254+0000 7f40f337e740 1 Processor -- start 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f97fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5632f9803200 con 0x5632f97f6f10 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5632f97157a0 con 0x5632f9800410 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5632f9803380 con 0x5632f97f7830 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40f337e740 10 client.0 _ll_register_callbacks cb 0x5632f9717b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40edc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40ee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f97fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40edc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53638/0 (socket says 172.21.15.7:53638) 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40edc00640 1 -- 172.21.15.7:0/3156720256 learned_addr learned my addr 172.21.15.7:0/3156720256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.255+0000 7f40ed200640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 -- 172.21.15.7:0/3156720256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 msgr2=0x5632f97fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 -- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 msgr2=0x5632f97fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f97fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 -- 172.21.15.7:0/3156720256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f95367e0 con 0x5632f97f6f10 2024-04-20T21:35:10.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40ee600640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f97fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:10.259 INFO:tasks.workunit.client.0.smithi007.stdout:checking for function prototypes... yes 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40edc00640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009900 tx=0x7f40dc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0178988d25a542 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40cb400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0be520 con 0x5632f97f6f10 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40dc0c6f30 con 0x5632f97f6f10 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.256+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d800fa20 con 0x5632f97f6f10 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40dc0bc890 con 0x5632f97f6f10 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 10 client.15651 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632f9596f70 con 0x5632f97f6f10 2024-04-20T21:35:10.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40d800fa20 con 0x5632f97f6f10 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f40dc0bdba0 con 0x5632f97f6f10 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 10 client.15651 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 20 client.15651 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9810740 0x5632f9812bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 4 client.15651 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.257+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632f9769740 con 0x5632f9810740 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.258+0000 7f40ed200640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9810740 0x5632f9812bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.258+0000 7f40ed200640 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9810740 0x5632f9812bc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6481a266e3951477 server_cookie=22b83a964f3cdba in_seq=0 out_seq=0 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.258+0000 7f40cb400640 10 client.15651 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40cb400640 1 -- 172.21.15.7:0/3156720256 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632f9769740 con 0x5632f9810740 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40cb400640 10 client.15651 handle_command_reply: tid=0 2024-04-20T21:35:10.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 client.15651 shutdown 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 msgr2=0x5632f97fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009900 tx=0x7f40dc003f90 comp rx=0 tx=0).stop 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 shutdown_connections 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f9800410 0x5632f97fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9810740 0x5632f9812bc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f7830 0x5632f97fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 --2- 172.21.15.7:0/3156720256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f97fd4a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40cb400640 0 client.15651 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 >> 172.21.15.7:0/3156720256 conn(0x5632f97f1640 msgr2=0x5632f97f3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 shutdown_connections 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.259+0000 7f40f337e740 1 -- 172.21.15.7:0/3156720256 wait complete. 2024-04-20T21:35:10.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.260+0000 7f40f337e740 20 client.15651 trim_cache size 0 max 16384 2024-04-20T21:35:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.279+0000 7f40f337e740 1 Processor -- start 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.279+0000 7f40f337e740 1 -- start start 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f97ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f7830 0x5632f9818490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ae220 con 0x5632f97f6f10 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81abae0 con 0x5632f97f7830 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40f337e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81a9950 con 0x5632f97f1640 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40ed200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40edc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f97ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40ed200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53650/0 (socket says 172.21.15.7:53650) 2024-04-20T21:35:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40ed200640 1 -- 172.21.15.7:0/1675870349 learned_addr learned my addr 172.21.15.7:0/1675870349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40ee600640 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f7830 0x5632f9818490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40edc00640 1 -- 172.21.15.7:0/1675870349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f7830 msgr2=0x5632f9818490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40edc00640 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f7830 0x5632f9818490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40edc00640 1 -- 172.21.15.7:0/1675870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 msgr2=0x5632f980c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.280+0000 7f40edc00640 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40edc00640 1 -- 172.21.15.7:0/1675870349 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632f942e170 con 0x5632f97f1640 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40ed200640 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40edc00640 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f97ffbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0be6e0 tx=0x7f40dc0ba360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a0231c3a44d966 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40d6000640 1 -- 172.21.15.7:0/1675870349 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0be0b0 con 0x5632f97f1640 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40d6000640 1 -- 172.21.15.7:0/1675870349 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc0bee50 con 0x5632f97f1640 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.281+0000 7f40d6000640 1 -- 172.21.15.7:0/1675870349 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0be290 con 0x5632f97f1640 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 -- 172.21.15.7:0/1675870349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 msgr2=0x5632f97ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f97ffbd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0be6e0 tx=0x7f40dc0ba360 comp rx=0 tx=0).stop 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 -- 172.21.15.7:0/1675870349 shutdown_connections 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f7830 0x5632f9818490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f6f10 0x5632f980c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 --2- 172.21.15.7:0/1675870349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f97ffbd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 -- 172.21.15.7:0/1675870349 >> 172.21.15.7:0/1675870349 conn(0x5632f97f8e70 msgr2=0x5632f97f3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 -- 172.21.15.7:0/1675870349 shutdown_connections 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.282+0000 7f40f337e740 1 -- 172.21.15.7:0/1675870349 wait complete. 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 -- start start 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 Processor -- start 2024-04-20T21:35:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f9808610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f7830 0x5632f9804b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81a9fa0 con 0x5632f97f7830 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ab410 con 0x5632f97f6f10 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81aa670 con 0x5632f97f1640 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.283+0000 7f40f337e740 10 client.0 _ll_register_callbacks cb 0x7f40dc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f7830 0x5632f9804b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40846/0 (socket says 172.21.15.7:40846) 2024-04-20T21:35:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 -- 172.21.15.7:0/3519632432 learned_addr learned my addr 172.21.15.7:0/3519632432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40edc00640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f9808610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 -- 172.21.15.7:0/3519632432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 msgr2=0x5632f9808610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f9808610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 -- 172.21.15.7:0/3519632432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f7830 msgr2=0x5632f9804b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f7830 0x5632f9804b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 -- 172.21.15.7:0/3519632432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632f9805390 con 0x5632f97f6f10 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40ed200640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5632f97f27d0 tx=0x7f40d80194a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eafd4ce38ba44db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40cb400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.284+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d802f070 con 0x5632f97f6f10 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc0bee50 con 0x5632f97f6f10 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40d800fcd0 con 0x5632f97f6f10 2024-04-20T21:35:10.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40d803b050 con 0x5632f97f6f10 2024-04-20T21:35:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 10 client.25772 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632f9877a10 con 0x5632f97f6f10 2024-04-20T21:35:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40dc0bee50 con 0x5632f97f6f10 2024-04-20T21:35:10.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f40d80196b0 con 0x5632f97f6f10 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 10 client.25772 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 20 client.25772 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9800410 0x5632f987a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 4 client.25772 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.285+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632f97635b0 con 0x5632f9800410 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.286+0000 7f40edc00640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9800410 0x5632f987a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.286+0000 7f40edc00640 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9800410 0x5632f987a090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df15f976bb0eb21 server_cookie=4326bc8ebbb0b82b in_seq=0 out_seq=0 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.286+0000 7f40cb400640 10 client.25772 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40cb400640 1 -- 172.21.15.7:0/3519632432 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632f97635b0 con 0x5632f9800410 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40cb400640 10 client.25772 handle_command_reply: tid=0 2024-04-20T21:35:10.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 client.25772 shutdown 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 msgr2=0x5632f9808b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5632f97f27d0 tx=0x7f40d80194a0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 shutdown_connections 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5632f97f6f10 0x5632f9808b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5632f9800410 0x5632f987a090 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5632f97f7830 0x5632f9804b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 --2- 172.21.15.7:0/3519632432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5632f97f1640 0x5632f9808610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40cb400640 0 client.25772 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 >> 172.21.15.7:0/3519632432 conn(0x5632f97f8e70 msgr2=0x5632f97f2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 shutdown_connections 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 1 -- 172.21.15.7:0/3519632432 wait complete. 2024-04-20T21:35:10.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.288+0000 7f40f337e740 20 client.25772 trim_cache size 0 max 16384 2024-04-20T21:35:10.292 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:35:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "fb3e5974-e7aa-4b0d-98a2-5a7bd476a1a9", 2024-04-20T21:35:10.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:35:10.292 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 msgr2=0x7f40d00aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 0x7f40d00aa6b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005f40 tx=0x7f40e00096a0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 msgr2=0x7f40e8196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8196690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009850 tx=0x7f40dc0042e0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 shutdown_connections 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40e819af50 0x7f40e819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40e8107260 0x7f40e8196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40d00a81f0 0x7f40d00aa6b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 --2- 172.21.15.7:0/2209447569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40e8102190 0x7f40e8196690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 >> 172.21.15.7:0/2209447569 conn(0x7f40e80fda10 msgr2=0x7f40e80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 shutdown_connections 2024-04-20T21:35:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.289+0000 7f40f0600640 1 -- 172.21.15.7:0/2209447569 wait complete. 2024-04-20T21:35:10.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:10.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:10.317 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether setvbuf arguments are reversed... no 2024-04-20T21:35:10.455 INFO:tasks.workunit.client.0.smithi007.stdout:checking for system... yes 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.512+0000 7f669de00640 1 -- 172.21.15.7:0/945812996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6698109470 msgr2=0x7f6698111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.512+0000 7f669de00640 1 --2- 172.21.15.7:0/945812996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6698109470 0x7f6698111040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6690009ca0 tx=0x7f669002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 -- 172.21.15.7:0/945812996 shutdown_connections 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 --2- 172.21.15.7:0/945812996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6698109470 0x7f6698111040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 --2- 172.21.15.7:0/945812996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6698104d50 0x7f6698108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 --2- 172.21.15.7:0/945812996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66981043a0 0x7f6698104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 -- 172.21.15.7:0/945812996 >> 172.21.15.7:0/945812996 conn(0x7f66980ffbe0 msgr2=0x7f6698102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 -- 172.21.15.7:0/945812996 shutdown_connections 2024-04-20T21:35:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.513+0000 7f669de00640 1 -- 172.21.15.7:0/945812996 wait complete. 2024-04-20T21:35:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 Processor -- start 2024-04-20T21:35:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 -- start start 2024-04-20T21:35:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 0x7f66981a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 0x7f66981a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 0x7f669819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6698114530 con 0x7f6698109470 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66981143b0 con 0x7f669819a8b0 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.514+0000 7f669de00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f66981146b0 con 0x7f66981043a0 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6696000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 0x7f66981a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6696a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 0x7f66981a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6696000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 0x7f66981a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53676/0 (socket says 172.21.15.7:53676) 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6696000640 1 -- 172.21.15.7:0/1623412789 learned_addr learned my addr 172.21.15.7:0/1623412789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 0x7f669819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 -- 172.21.15.7:0/1623412789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 msgr2=0x7f66981a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 0x7f66981a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 -- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 msgr2=0x7f66981a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 0x7f66981a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6697400640 1 -- 172.21.15.7:0/1623412789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819b3d0 con 0x7f669819a8b0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.515+0000 7f6696a00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 0x7f66981a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f6697400640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 0x7f669819ad60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6690009bf0 tx=0x7f6690035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f669ca00640 1 -- 172.21.15.7:0/1623412789 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690039040 con 0x7f669819a8b0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f669ca00640 1 -- 172.21.15.7:0/1623412789 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6690044070 con 0x7f669819a8b0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819b660 con 0x7f669819a8b0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f669ca00640 1 -- 172.21.15.7:0/1623412789 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900042a0 con 0x7f669819a8b0 2024-04-20T21:35:10.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.516+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a7850 con 0x7f669819a8b0 2024-04-20T21:35:10.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.519+0000 7f669ca00640 1 -- 172.21.15.7:0/1623412789 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6690004440 con 0x7f669819a8b0 2024-04-20T21:35:10.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.519+0000 7f66a0ba8740 1 Processor -- start 2024-04-20T21:35:10.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 -- start start 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 0x5597fb1d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5597fb134150 con 0x5597fb1ce1a0 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5597fb0da370 con 0x5597fb1ceac0 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5597fb130110 con 0x5597fb1d76a0 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.520+0000 7f6696000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53692/0 (socket says 172.21.15.7:53692) 2024-04-20T21:35:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696000640 1 -- 172.21.15.7:0/2572972502 learned_addr learned my addr 172.21.15.7:0/2572972502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696000640 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 0x5597fb1d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 -- 172.21.15.7:0/2572972502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 msgr2=0x5597fb1d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 0x5597fb1d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 -- 172.21.15.7:0/2572972502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 msgr2=0x5597fb1ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 -- 172.21.15.7:0/2572972502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597faf107d0 con 0x5597fb1ceac0 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696000640 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 0x5597fb1d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f669ca00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 0x7f667c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f6696a00640 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1d7160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fa0 tx=0x7f66880118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3814048a745077c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.521+0000 7f669ca00640 1 -- 172.21.15.7:0/1623412789 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f66900f2ba0 con 0x7f669819a8b0 2024-04-20T21:35:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.522+0000 7f6696000640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 0x7f667c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.522+0000 7f6683e00640 1 -- 172.21.15.7:0/2572972502 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004030 con 0x5597fb1ceac0 2024-04-20T21:35:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.522+0000 7f6683e00640 1 -- 172.21.15.7:0/2572972502 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688005500 con 0x5597fb1ceac0 2024-04-20T21:35:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.522+0000 7f6683e00640 1 -- 172.21.15.7:0/2572972502 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688004e10 con 0x5597fb1ceac0 2024-04-20T21:35:10.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f6696000640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 0x7f667c0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5597fb1c9a60 tx=0x7f6684009820 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- 172.21.15.7:0/2572972502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 msgr2=0x5597fb1d7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1d7160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6688009fa0 tx=0x7f66880118d0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- 172.21.15.7:0/2572972502 shutdown_connections 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1d7160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1d76a0 0x5597fb1d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/2572972502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- 172.21.15.7:0/2572972502 >> 172.21.15.7:0/2572972502 conn(0x5597fb1c88d0 msgr2=0x5597fb1c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- 172.21.15.7:0/2572972502 shutdown_connections 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- 172.21.15.7:0/2572972502 wait complete. 2024-04-20T21:35:10.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.523+0000 7f66a0ba8740 1 -- start start 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 Processor -- start 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5597fb1da490 con 0x5597fb1d76a0 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5597fb0ed740 con 0x5597fb1ce1a0 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5597fb1da610 con 0x5597fb1ceac0 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f66a0ba8740 10 client.0 _ll_register_callbacks cb 0x5597fb0e8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f6696000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f6696000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44638/0 (socket says 172.21.15.7:44638) 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f6696000640 1 -- 172.21.15.7:0/3949305841 learned_addr learned my addr 172.21.15.7:0/3949305841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.524+0000 7f6696a00640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6697400640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 -- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 msgr2=0x5597fb1d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 -- 172.21.15.7:0/3949305841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 msgr2=0x5597fb1d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 -- 172.21.15.7:0/3949305841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fae463f0 con 0x5597fb1ceac0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696a00640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6696000640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f668401fa30 tx=0x7f6684004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11076a3cb781fe1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.525+0000 7f6680c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668402f060 con 0x5597fb1ceac0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6688005500 con 0x5597fb1ceac0 2024-04-20T21:35:10.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6684019030 con 0x5597fb1ceac0 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6684034460 con 0x5597fb1ceac0 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f66a0ba8740 10 client.25777 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597fae060f0 con 0x5597fb1ceac0 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.526+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6688005500 con 0x5597fb1ceac0 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f668400f660 con 0x5597fb1ceac0 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 10 client.25777 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 10 client.25777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 20 client.25777 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb1e7a30 0x5597fb1e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 4 client.25777 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fb141b70 con 0x5597fb1e7a30 2024-04-20T21:35:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.527+0000 7f6696a00640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb1e7a30 0x5597fb1e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.528+0000 7f6696a00640 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb1e7a30 0x5597fb1e9ec0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188cb21997cee2b4 server_cookie=e08fd5f1841224ca in_seq=0 out_seq=0 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.528+0000 7f6680c00640 10 client.25777 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.528+0000 7f6680c00640 1 -- 172.21.15.7:0/3949305841 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597fb141b70 con 0x5597fb1e7a30 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.528+0000 7f6680c00640 10 client.25777 handle_command_reply: tid=0 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.528+0000 7f66a0ba8740 1 client.25777 shutdown 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 msgr2=0x5597fb1d4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f668401fa30 tx=0x7f6684004730 comp rx=0 tx=0).stop 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 shutdown_connections 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb1e7a30 0x5597fb1e9ec0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1d4c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f6680c00640 0 client.25777 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/3949305841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 >> 172.21.15.7:0/3949305841 conn(0x5597fb1c88d0 msgr2=0x5597fb1ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 shutdown_connections 2024-04-20T21:35:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 1 -- 172.21.15.7:0/3949305841 wait complete. 2024-04-20T21:35:10.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.529+0000 7f66a0ba8740 20 client.25777 trim_cache size 0 max 16384 2024-04-20T21:35:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 Processor -- start 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 -- start start 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66981b2690 con 0x5597fb1ce1a0 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66981aff50 con 0x5597fb1d76a0 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f66981addc0 con 0x5597fb1ceac0 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f6696a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f6697400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f6696a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53702/0 (socket says 172.21.15.7:53702) 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f6697400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40880/0 (socket says 172.21.15.7:40880) 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.549+0000 7f6696a00640 1 -- 172.21.15.7:0/362314206 learned_addr learned my addr 172.21.15.7:0/362314206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6696000640 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 -- 172.21.15.7:0/362314206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 msgr2=0x5597fb1e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 -- 172.21.15.7:0/362314206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 msgr2=0x5597fb1d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 -- 172.21.15.7:0/362314206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fb1d2c40 con 0x5597fb1d76a0 2024-04-20T21:35:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6697400640 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66900f0400 tx=0x7f66900be5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec00a13e8aefd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6683400640 1 -- 172.21.15.7:0/362314206 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900c3030 con 0x5597fb1d76a0 2024-04-20T21:35:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6683400640 1 -- 172.21.15.7:0/362314206 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66900c78f0 con 0x5597fb1d76a0 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.550+0000 7f6683400640 1 -- 172.21.15.7:0/362314206 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900beee0 con 0x5597fb1d76a0 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 -- 172.21.15.7:0/362314206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 msgr2=0x5597fb1ef7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f66900f0400 tx=0x7f66900be5e0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 -- 172.21.15.7:0/362314206 shutdown_connections 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1d76a0 0x5597fb1ef7d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ceac0 0x5597fb1e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/362314206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1ce1a0 0x5597fb1d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.551+0000 7f66a0ba8740 1 -- 172.21.15.7:0/362314206 >> 172.21.15.7:0/362314206 conn(0x5597fb1c88d0 msgr2=0x5597fb1d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.552+0000 7f66a0ba8740 1 -- 172.21.15.7:0/362314206 shutdown_connections 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.552+0000 7f66a0ba8740 1 -- 172.21.15.7:0/362314206 wait complete. 2024-04-20T21:35:10.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.552+0000 7f66a0ba8740 1 -- start start 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.552+0000 7f66a0ba8740 1 Processor -- start 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 0x5597fb1df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ae410 con 0x5597fb1d76a0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f66981af880 con 0x5597fb1ceac0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f66981aeae0 con 0x5597fb1ce1a0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f66a0ba8740 10 client.0 _ll_register_callbacks cb 0x7f6684012f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6697400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40896/0 (socket says 172.21.15.7:40896) 2024-04-20T21:35:10.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 -- 172.21.15.7:0/1823765526 learned_addr learned my addr 172.21.15.7:0/1823765526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696a00640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 0x5597fb1df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 -- 172.21.15.7:0/1823765526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 msgr2=0x5597fb1df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 0x5597fb1df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 -- 172.21.15.7:0/1823765526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 msgr2=0x5597fb1d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696000640 1 -- 172.21.15.7:0/1823765526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fb1d1750 con 0x5597fb1ceac0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.553+0000 7f6696a00640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 0x5597fb1df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f6696000640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f66840095e0 tx=0x7f668400f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd5e92bd629baa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f6680c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668401f7c0 con 0x5597fb1ceac0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66900c78f0 con 0x5597fb1ceac0 2024-04-20T21:35:10.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6684009e40 con 0x5597fb1ceac0 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6684034810 con 0x5597fb1ceac0 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f66a0ba8740 10 client.25790 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.554+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597fb24dc60 con 0x5597fb1ceac0 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66900c78f0 con 0x5597fb1ceac0 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f668403d020 con 0x5597fb1ceac0 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 10 client.25790 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 20 client.25790 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb24e420 0x5597fb250840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 4 client.25790 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597fb13b9e0 con 0x5597fb24e420 2024-04-20T21:35:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.555+0000 7f6696a00640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb24e420 0x5597fb250840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f6696a00640 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb24e420 0x5597fb250840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47915fd1940f3711 server_cookie=faed6fb408cf398e in_seq=0 out_seq=0 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f6680c00640 10 client.25790 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f6680c00640 1 -- 172.21.15.7:0/1823765526 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5597fb13b9e0 con 0x5597fb24e420 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f6680c00640 10 client.25790 handle_command_reply: tid=0 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 client.25790 shutdown 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 msgr2=0x5597fb1dfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f66840095e0 tx=0x7f668400f670 comp rx=0 tx=0).stop 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 shutdown_connections 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5597fb1ceac0 0x5597fb1dfdd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5597fb24e420 0x5597fb250840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5597fb1d76a0 0x5597fb1d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 --2- 172.21.15.7:0/1823765526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5597fb1ce1a0 0x5597fb1df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 >> 172.21.15.7:0/1823765526 conn(0x5597fb1c88d0 msgr2=0x5597fb1c9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f6680c00640 0 client.25790 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 shutdown_connections 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.556+0000 7f66a0ba8740 1 -- 172.21.15.7:0/1823765526 wait complete. 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f66a0ba8740 20 client.25790 trim_cache size 0 max 16384 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:35:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: "fb3e5974-e7aa-4b0d-98a2-5a7bd476a1a9": { 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "fb3e5974-e7aa-4b0d-98a2-5a7bd476a1a9", 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:35:10.562 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 msgr2=0x7f667c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 0x7f667c0a61e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5597fb1c9a60 tx=0x7f6684009820 comp rx=0 tx=0).stop 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 msgr2=0x7f669819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 0x7f669819ad60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6690009bf0 tx=0x7f6690035d10 comp rx=0 tx=0).stop 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 shutdown_connections 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f667c0a3d20 0x7f667c0a61e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f669819a8b0 0x7f669819ad60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6698109470 0x7f66981a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 --2- 172.21.15.7:0/1623412789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f66981043a0 0x7f66981a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.557+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 >> 172.21.15.7:0/1623412789 conn(0x7f66980ffbe0 msgr2=0x7f66981007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.558+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 shutdown_connections 2024-04-20T21:35:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:10.558+0000 7f669de00640 1 -- 172.21.15.7:0/1623412789 wait complete. 2024-04-20T21:35:10.575 INFO:tasks.cephfs.filesystem:scrub status for tag:fb3e5974-e7aa-4b0d-98a2-5a7bd476a1a9 - {'path': '/', 'tag': 'fb3e5974-e7aa-4b0d-98a2-5a7bd476a1a9', 'options': 'recursive,force'} 2024-04-20T21:35:10.590 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gettimeofday... yes 2024-04-20T21:35:10.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:10 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:10.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:10 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:10 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:10 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:10.746 INFO:tasks.workunit.client.0.smithi007.stdout:checking for mkdir... yes 2024-04-20T21:35:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:10 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:10 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:10.901 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strerror... yes 2024-04-20T21:35:11.058 INFO:tasks.workunit.client.0.smithi007.stdout:checking for open64... yes 2024-04-20T21:35:11.197 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stat64... yes 2024-04-20T21:35:11.351 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fseeko64... yes 2024-04-20T21:35:11.513 INFO:tasks.workunit.client.0.smithi007.stdout:checking for lrand48_r... yes 2024-04-20T21:35:11.656 INFO:tasks.workunit.client.0.smithi007.stdout:checking for srand48_r... yes 2024-04-20T21:35:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:35:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:11 smithi016 ceph-mon[21409]: pgmap v58: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:11.713 INFO:tasks.workunit.client.0.smithi007.stdout:configure: creating ./config.status 2024-04-20T21:35:11.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:35:11.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:11.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:11 smithi186 ceph-mon[22157]: pgmap v58: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:11.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:35:11.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:11.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:11.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:11 smithi007 ceph-mon[16867]: pgmap v58: 97 pgs: 97 active+clean; 3.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 177 KiB/s wr, 28 op/s 2024-04-20T21:35:12.020 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating Makefile 2024-04-20T21:35:12.067 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating config.h 2024-04-20T21:35:12.124 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: executing depfiles commands 2024-04-20T21:35:12.265 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: executing default commands 2024-04-20T21:35:12.356 INFO:tasks.workunit.client.0.smithi007.stderr:+ make 2024-04-20T21:35:12.369 INFO:tasks.workunit.client.0.smithi007.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-20T21:35:12.376 INFO:tasks.workunit.client.0.smithi007.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-20T21:35:12.380 INFO:tasks.workunit.client.0.smithi007.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-20T21:35:12.380 INFO:tasks.workunit.client.0.smithi007.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-20T21:35:12.380 INFO:tasks.workunit.client.0.smithi007.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-20T21:35:12.380 INFO:tasks.workunit.client.0.smithi007.stderr: any GNU archive site. 2024-04-20T21:35:12.387 INFO:tasks.workunit.client.0.smithi007.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-20T21:35:12.392 INFO:tasks.workunit.client.0.smithi007.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-20T21:35:12.395 INFO:tasks.workunit.client.0.smithi007.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-20T21:35:12.395 INFO:tasks.workunit.client.0.smithi007.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-20T21:35:12.395 INFO:tasks.workunit.client.0.smithi007.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-20T21:35:12.395 INFO:tasks.workunit.client.0.smithi007.stderr: Grab them from any GNU archive site. 2024-04-20T21:35:12.496 INFO:tasks.workunit.client.0.smithi007.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-20T21:35:13.025 INFO:tasks.workunit.client.0.smithi007.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-20T21:35:13.026 INFO:tasks.workunit.client.0.smithi007.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-20T21:35:13.026 INFO:tasks.workunit.client.0.smithi007.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-20T21:35:13.026 INFO:tasks.workunit.client.0.smithi007.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-20T21:35:13.228 INFO:tasks.workunit.client.0.smithi007.stdout:/bin/sh ./config.status --recheck 2024-04-20T21:35:13.398 INFO:tasks.workunit.client.0.smithi007.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-20T21:35:13.699 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-20T21:35:14.397 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:14 smithi016 ceph-mon[21409]: pgmap v59: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 414 KiB/s wr, 60 op/s 2024-04-20T21:35:14.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:14 smithi186 ceph-mon[22157]: pgmap v59: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 414 KiB/s wr, 60 op/s 2024-04-20T21:35:14.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:14 smithi007 ceph-mon[16867]: pgmap v59: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 414 KiB/s wr, 60 op/s 2024-04-20T21:35:14.720 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether build environment is sane... yes 2024-04-20T21:35:14.737 INFO:tasks.workunit.client.0.smithi007.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-20T21:35:14.738 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gawk... gawk 2024-04-20T21:35:14.770 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether make sets $(MAKE)... yes 2024-04-20T21:35:14.866 INFO:tasks.workunit.client.0.smithi007.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-20T21:35:14.867 INFO:tasks.workunit.client.0.smithi007.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-20T21:35:14.887 INFO:tasks.workunit.client.0.smithi007.stdout:checking for style of include used by make... GNU 2024-04-20T21:35:14.894 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... gcc 2024-04-20T21:35:15.098 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether the C compiler works... yes 2024-04-20T21:35:15.098 INFO:tasks.workunit.client.0.smithi007.stdout:checking for C compiler default output file name... a.out 2024-04-20T21:35:15.187 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of executables... 2024-04-20T21:35:15.283 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are cross compiling... no 2024-04-20T21:35:15.364 INFO:tasks.workunit.client.0.smithi007.stdout:checking for suffix of object files... o 2024-04-20T21:35:15.426 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... yes 2024-04-20T21:35:15.488 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... yes 2024-04-20T21:35:15.573 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-20T21:35:15.775 INFO:tasks.workunit.client.0.smithi007.stdout:checking dependency style of gcc... gcc3 2024-04-20T21:35:15.882 INFO:tasks.workunit.client.0.smithi007.stdout:checking how to run the C preprocessor... gcc -E 2024-04-20T21:35:16.000 INFO:tasks.workunit.client.0.smithi007.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-20T21:35:16.006 INFO:tasks.workunit.client.0.smithi007.stdout:checking for egrep... /usr/bin/grep -E 2024-04-20T21:35:16.286 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ANSI C header files... yes 2024-04-20T21:35:16.365 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/types.h... yes 2024-04-20T21:35:16.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:16 smithi016 ceph-mon[21409]: pgmap v60: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 391 KiB/s wr, 43 op/s 2024-04-20T21:35:16.458 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/stat.h... yes 2024-04-20T21:35:16.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:16 smithi186 ceph-mon[22157]: pgmap v60: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 391 KiB/s wr, 43 op/s 2024-04-20T21:35:16.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:16 smithi007 ceph-mon[16867]: pgmap v60: 97 pgs: 97 active+clean; 5.6 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 391 KiB/s wr, 43 op/s 2024-04-20T21:35:16.554 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdlib.h... yes 2024-04-20T21:35:16.654 INFO:tasks.workunit.client.0.smithi007.stdout:checking for string.h... yes 2024-04-20T21:35:16.751 INFO:tasks.workunit.client.0.smithi007.stdout:checking for memory.h... yes 2024-04-20T21:35:16.847 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strings.h... yes 2024-04-20T21:35:16.944 INFO:tasks.workunit.client.0.smithi007.stdout:checking for inttypes.h... yes 2024-04-20T21:35:17.042 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdint.h... yes 2024-04-20T21:35:17.134 INFO:tasks.workunit.client.0.smithi007.stdout:checking for unistd.h... yes 2024-04-20T21:35:17.226 INFO:tasks.workunit.client.0.smithi007.stdout:checking minix/config.h usability... no 2024-04-20T21:35:17.288 INFO:tasks.workunit.client.0.smithi007.stdout:checking minix/config.h presence... no 2024-04-20T21:35:17.288 INFO:tasks.workunit.client.0.smithi007.stdout:checking for minix/config.h... no 2024-04-20T21:35:17.373 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-20T21:35:17.374 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc... (cached) gcc 2024-04-20T21:35:17.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:17.455 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-20T21:35:17.455 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-20T21:35:17.455 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-20T21:35:17.456 INFO:tasks.workunit.client.0.smithi007.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-20T21:35:17.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:17.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:17.560 INFO:tasks.workunit.client.0.smithi007.stdout:checking for main in -lm... yes 2024-04-20T21:35:17.668 INFO:tasks.workunit.client.0.smithi007.stdout:checking for main in -lpthread... yes 2024-04-20T21:35:17.672 INFO:tasks.workunit.client.0.smithi007.stdout:checking for ANSI C header files... (cached) yes 2024-04-20T21:35:17.750 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-20T21:35:17.872 INFO:tasks.workunit.client.0.smithi007.stdout:checking pthread.h usability... yes 2024-04-20T21:35:17.918 INFO:tasks.workunit.client.0.smithi007.stdout:checking pthread.h presence... yes 2024-04-20T21:35:17.919 INFO:tasks.workunit.client.0.smithi007.stdout:checking for pthread.h... yes 2024-04-20T21:35:18.031 INFO:tasks.workunit.client.0.smithi007.stdout:checking fcntl.h usability... yes 2024-04-20T21:35:18.074 INFO:tasks.workunit.client.0.smithi007.stdout:checking fcntl.h presence... yes 2024-04-20T21:35:18.074 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fcntl.h... yes 2024-04-20T21:35:18.187 INFO:tasks.workunit.client.0.smithi007.stdout:checking limits.h usability... yes 2024-04-20T21:35:18.227 INFO:tasks.workunit.client.0.smithi007.stdout:checking limits.h presence... yes 2024-04-20T21:35:18.227 INFO:tasks.workunit.client.0.smithi007.stdout:checking for limits.h... yes 2024-04-20T21:35:18.260 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stdint.h... (cached) yes 2024-04-20T21:35:18.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:18 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:18.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:18 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:18.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:18 smithi016 ceph-mon[21409]: pgmap v61: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 478 KiB/s wr, 51 op/s 2024-04-20T21:35:18.463 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h usability... yes 2024-04-20T21:35:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:18 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:18 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:18.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:18 smithi186 ceph-mon[22157]: pgmap v61: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 478 KiB/s wr, 51 op/s 2024-04-20T21:35:18.496 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/time.h presence... yes 2024-04-20T21:35:18.497 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/time.h... yes 2024-04-20T21:35:18.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:18 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:18.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:18 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:18.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:18 smithi007 ceph-mon[16867]: pgmap v61: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 478 KiB/s wr, 51 op/s 2024-04-20T21:35:18.543 INFO:tasks.workunit.client.0.smithi007.stdout:checking for unistd.h... (cached) yes 2024-04-20T21:35:18.634 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/vfs.h usability... yes 2024-04-20T21:35:18.672 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/vfs.h presence... yes 2024-04-20T21:35:18.672 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/vfs.h... yes 2024-04-20T21:35:18.783 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/limits.h usability... no 2024-04-20T21:35:18.847 INFO:tasks.workunit.client.0.smithi007.stdout:checking sys/limits.h presence... no 2024-04-20T21:35:18.848 INFO:tasks.workunit.client.0.smithi007.stdout:checking for sys/limits.h... no 2024-04-20T21:35:18.930 INFO:tasks.workunit.client.0.smithi007.stdout:checking for an ANSI C-conforming const... yes 2024-04-20T21:35:18.998 INFO:tasks.workunit.client.0.smithi007.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-20T21:35:19.141 INFO:tasks.workunit.client.0.smithi007.stdout:checking for system... yes 2024-04-20T21:35:19.272 INFO:tasks.workunit.client.0.smithi007.stdout:checking for gettimeofday... yes 2024-04-20T21:35:19.397 INFO:tasks.workunit.client.0.smithi007.stdout:checking for mkdir... yes 2024-04-20T21:35:19.533 INFO:tasks.workunit.client.0.smithi007.stdout:checking for strerror... yes 2024-04-20T21:35:19.669 INFO:tasks.workunit.client.0.smithi007.stdout:checking for open64... yes 2024-04-20T21:35:19.799 INFO:tasks.workunit.client.0.smithi007.stdout:checking for stat64... yes 2024-04-20T21:35:19.936 INFO:tasks.workunit.client.0.smithi007.stdout:checking for fseeko64... yes 2024-04-20T21:35:20.071 INFO:tasks.workunit.client.0.smithi007.stdout:checking for lrand48_r... yes 2024-04-20T21:35:20.205 INFO:tasks.workunit.client.0.smithi007.stdout:checking for srand48_r... yes 2024-04-20T21:35:20.257 INFO:tasks.workunit.client.0.smithi007.stdout:configure: creating ./config.status 2024-04-20T21:35:20.422 INFO:tasks.workunit.client.0.smithi007.stdout: /bin/sh ./config.status 2024-04-20T21:35:20.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:20 smithi016 ceph-mon[21409]: pgmap v62: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:20.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:20 smithi186 ceph-mon[22157]: pgmap v62: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:20.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:20 smithi007 ceph-mon[16867]: pgmap v62: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:20.561 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating Makefile 2024-04-20T21:35:20.601 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating config.h 2024-04-20T21:35:20.654 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: executing depfiles commands 2024-04-20T21:35:20.700 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: executing default commands 2024-04-20T21:35:20.755 INFO:tasks.workunit.client.0.smithi007.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-20T21:35:21.022 INFO:tasks.workunit.client.0.smithi007.stdout:rm -f stamp-h1 2024-04-20T21:35:21.026 INFO:tasks.workunit.client.0.smithi007.stdout:touch config.h.in 2024-04-20T21:35:21.031 INFO:tasks.workunit.client.0.smithi007.stdout:cd . && /bin/sh ./config.status config.h 2024-04-20T21:35:21.153 INFO:tasks.workunit.client.0.smithi007.stdout:config.status: creating config.h 2024-04-20T21:35:21.204 INFO:tasks.workunit.client.0.smithi007.stdout:make all-am 2024-04-20T21:35:21.213 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-20T21:35:21.213 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-20T21:35:21.627 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-20T21:35:21.633 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-20T21:35:21.921 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-20T21:35:21.929 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-20T21:35:22.086 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-20T21:35:22.092 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-20T21:35:22.232 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-20T21:35:22.239 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-20T21:35:22.414 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-20T21:35:22.420 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-20T21:35:22.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:22 smithi016 ceph-mon[21409]: pgmap v63: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:22.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:22 smithi186 ceph-mon[22157]: pgmap v63: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-20T21:35:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-20T21:35:22.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:22 smithi007 ceph-mon[16867]: pgmap v63: 97 pgs: 97 active+clean; 6.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 324 KiB/s wr, 40 op/s 2024-04-20T21:35:22.603 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-20T21:35:22.611 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-20T21:35:22.701 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-20T21:35:22.707 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-20T21:35:22.979 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-20T21:35:22.985 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-20T21:35:23.164 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-20T21:35:23.171 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-20T21:35:23.399 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-20T21:35:23.406 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-20T21:35:23.523 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-20T21:35:23.530 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-20T21:35:23.648 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-20T21:35:23.654 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-20T21:35:23.816 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-20T21:35:23.823 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-20T21:35:23.879 INFO:tasks.workunit.client.0.smithi007.stderr:In file included from parser.h:22, 2024-04-20T21:35:23.879 INFO:tasks.workunit.client.0.smithi007.stderr: from parser.c:25: 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c: In function ‘init_filesys’: 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-20T21:35:23.880 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.881 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.882 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.883 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.884 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.885 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-20T21:35:23.886 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.887 INFO:tasks.workunit.client.0.smithi007.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: | ^ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: 36 | container_of(ptr, type, member) 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-20T21:35:23.888 INFO:tasks.workunit.client.0.smithi007.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-20T21:35:23.889 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c: In function ‘get_optstr’: 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:200:14: note: source object declared here 2024-04-20T21:35:23.994 INFO:tasks.workunit.client.0.smithi007.stderr: 200 | char temp[BUFSIZE]; 2024-04-20T21:35:23.995 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~ 2024-04-20T21:35:24.002 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c: In function ‘size64_convert’: 2024-04-20T21:35:24.002 INFO:tasks.workunit.client.0.smithi007.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-20T21:35:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-20T21:35:24.003 INFO:tasks.workunit.client.0.smithi007.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-20T21:35:24.371 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-20T21:35:24.378 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-20T21:35:24.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:24 smithi016 ceph-mon[21409]: pgmap v64: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 541 KiB/s wr, 82 op/s 2024-04-20T21:35:24.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:24 smithi186 ceph-mon[22157]: pgmap v64: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 541 KiB/s wr, 82 op/s 2024-04-20T21:35:24.480 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-20T21:35:24.486 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-20T21:35:24.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:24 smithi007 ceph-mon[16867]: pgmap v64: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 541 KiB/s wr, 82 op/s 2024-04-20T21:35:24.629 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-20T21:35:24.636 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-20T21:35:24.702 INFO:tasks.workunit.client.0.smithi007.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-20T21:35:24.711 INFO:tasks.workunit.client.0.smithi007.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-20T21:35:24.784 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-20T21:35:24.786 INFO:tasks.workunit.client.0.smithi007.stderr:+ cd .. 2024-04-20T21:35:24.786 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir tmp 2024-04-20T21:35:24.791 INFO:tasks.workunit.client.0.smithi007.stderr:+ cd tmp 2024-04-20T21:35:24.792 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in $mydir/*.ffsb 2024-04-20T21:35:24.792 INFO:tasks.workunit.client.0.smithi007.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-20T21:35:24.794 INFO:tasks.workunit.client.0.smithi007.stdout:FFSB version 6.0-RC2 started 2024-04-20T21:35:24.795 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:benchmark time = 300 2024-04-20T21:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:ThreadGroup 0 2024-04-20T21:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout:================ 2024-04-20T21:35:24.803 INFO:tasks.workunit.client.0.smithi007.stdout: num_threads = 32 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: read_random = off 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: read_size = 0 (0B) 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: read_blocksize = 0 (0B) 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: read_skip = off 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: read_skipsize = 0 (0B) 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: write_random = on 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: write_size = 5242880 (5MB) 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: fsync_file = 0 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: write_blocksize = 4096 (4KB) 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: wait time = 0 2024-04-20T21:35:24.804 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: op weights 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: read = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: readall = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: write = 1 (100.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: create = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: append = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: delete = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: metaop = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: createdir = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: stat = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: writeall = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: writeall_fsync = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: open_close = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: write_fsync = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: create_fsync = 0 (0.00%) 2024-04-20T21:35:24.805 INFO:tasks.workunit.client.0.smithi007.stdout: append_fsync = 0 (0.00%) 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout:FileSystem . 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout:========== 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: num_dirs = 0 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: starting files = 128 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: min file size = 104857600 (100MB) 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: max file size = 104857600 (100MB) 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: directio = off 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: alignedio = on 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: bufferedio = off 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.806 INFO:tasks.workunit.client.0.smithi007.stdout: aging is off 2024-04-20T21:35:24.807 INFO:tasks.workunit.client.0.smithi007.stdout: current utilization = 0.00% 2024-04-20T21:35:24.807 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:35:24.807 INFO:tasks.workunit.client.0.smithi007.stdout:checking existing fs: . 2024-04-20T21:35:24.807 INFO:tasks.workunit.client.0.smithi007.stderr:opendir: No such file or directory 2024-04-20T21:35:24.808 INFO:tasks.workunit.client.0.smithi007.stdout:recreating new fileset 2024-04-20T21:35:26.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:26 smithi016 ceph-mon[21409]: pgmap v65: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 304 KiB/s wr, 50 op/s 2024-04-20T21:35:26.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:26 smithi186 ceph-mon[22157]: pgmap v65: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 304 KiB/s wr, 50 op/s 2024-04-20T21:35:26.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:26 smithi007 ceph-mon[16867]: pgmap v65: 97 pgs: 97 active+clean; 8.1 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 304 KiB/s wr, 50 op/s 2024-04-20T21:35:28.445 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:28 smithi016 ceph-mon[21409]: pgmap v66: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 344 KiB/s wr, 59 op/s 2024-04-20T21:35:28.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:28 smithi186 ceph-mon[22157]: pgmap v66: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 344 KiB/s wr, 59 op/s 2024-04-20T21:35:28.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:28 smithi007 ceph-mon[16867]: pgmap v66: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 344 KiB/s wr, 59 op/s 2024-04-20T21:35:30.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:30 smithi016 ceph-mon[21409]: pgmap v67: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:30.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:30 smithi186 ceph-mon[22157]: pgmap v67: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:30.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:30 smithi007 ceph-mon[16867]: pgmap v67: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:32.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:32 smithi016 ceph-mon[21409]: pgmap v68: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:32.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:32 smithi186 ceph-mon[22157]: pgmap v68: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:32.541 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:32 smithi007 ceph-mon[16867]: pgmap v68: 97 pgs: 97 active+clean; 8.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 258 KiB/s wr, 50 op/s 2024-04-20T21:35:33.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:33 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:35:33.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:33 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:35:33.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:33 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:35:34.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:34 smithi016 ceph-mon[21409]: pgmap v69: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 82 op/s 2024-04-20T21:35:34.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:34 smithi186 ceph-mon[22157]: pgmap v69: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 82 op/s 2024-04-20T21:35:34.537 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:34 smithi007 ceph-mon[16867]: pgmap v69: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 9.1 MiB/s wr, 82 op/s 2024-04-20T21:35:35.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:35 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:35.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:35 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:35.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:35 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:35:35.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:35 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:35.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:35 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:35.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:35 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:35:35.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:35 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:35:35.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:35 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:35:35.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:35 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:35:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:36 smithi016 ceph-mon[21409]: pgmap v70: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.9 MiB/s wr, 41 op/s 2024-04-20T21:35:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:36 smithi186 ceph-mon[22157]: pgmap v70: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.9 MiB/s wr, 41 op/s 2024-04-20T21:35:36.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:36 smithi007 ceph-mon[16867]: pgmap v70: 97 pgs: 97 active+clean; 114 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.9 MiB/s wr, 41 op/s 2024-04-20T21:35:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:38 smithi016 ceph-mon[21409]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 127 MiB/s wr, 82 op/s 2024-04-20T21:35:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:38 smithi186 ceph-mon[22157]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 127 MiB/s wr, 82 op/s 2024-04-20T21:35:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:38 smithi007 ceph-mon[16867]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 127 MiB/s wr, 82 op/s 2024-04-20T21:35:40.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:40.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:40.662 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:40 smithi007 ceph-mon[16867]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:40 smithi016 ceph-mon[21409]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:40 smithi186 ceph-mon[22157]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 -- 172.21.15.7:0/546052949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109a60 msgr2=0x7f2e5c114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 --2- 172.21.15.7:0/546052949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109a60 0x7f2e5c114b70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009890 tx=0x7f2e4802bb90 comp rx=0 tx=0).stop 2024-04-20T21:35:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 -- 172.21.15.7:0/546052949 shutdown_connections 2024-04-20T21:35:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 --2- 172.21.15.7:0/546052949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 0x7f2e5c1174a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 --2- 172.21.15.7:0/546052949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109a60 0x7f2e5c114b70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 --2- 172.21.15.7:0/546052949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109140 0x7f2e5c109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.955+0000 7f2e62a00640 1 -- 172.21.15.7:0/546052949 >> 172.21.15.7:0/546052949 conn(0x7f2e5c06d000 msgr2=0x7f2e5c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.957+0000 7f2e62a00640 1 -- 172.21.15.7:0/546052949 shutdown_connections 2024-04-20T21:35:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.957+0000 7f2e62a00640 1 -- 172.21.15.7:0/546052949 wait complete. 2024-04-20T21:35:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 Processor -- start 2024-04-20T21:35:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 -- start start 2024-04-20T21:35:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109a60 0x7f2e5c1a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 0x7f2e5c1abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c121100 con 0x7f2e5c109a60 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c120f80 con 0x7f2e5c1150b0 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.958+0000 7f2e62a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c121280 con 0x7f2e5c109140 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55386/0 (socket says 172.21.15.7:55386) 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 -- 172.21.15.7:0/3670237947 learned_addr learned my addr 172.21.15.7:0/3670237947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5b400640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109a60 0x7f2e5c1a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e60a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 0x7f2e5c1abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 -- 172.21.15.7:0/3670237947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 msgr2=0x7f2e5c1abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 0x7f2e5c1abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 -- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109a60 msgr2=0x7f2e5c1a7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109a60 0x7f2e5c1a7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.959+0000 7f2e5be00640 1 -- 172.21.15.7:0/3670237947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c1ac6d0 con 0x7f2e5c109140 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.960+0000 7f2e5be00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e9f0 tx=0x7f2e4c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.960+0000 7f2e58c00640 1 -- 172.21.15.7:0/3670237947 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00cd80 con 0x7f2e5c109140 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.961+0000 7f2e58c00640 1 -- 172.21.15.7:0/3670237947 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e4c010910 con 0x7f2e5c109140 2024-04-20T21:35:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.961+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c1ac9c0 con 0x7f2e5c109140 2024-04-20T21:35:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.961+0000 7f2e58c00640 1 -- 172.21.15.7:0/3670237947 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0188f0 con 0x7f2e5c109140 2024-04-20T21:35:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.961+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1b4250 con 0x7f2e5c109140 2024-04-20T21:35:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.967+0000 7f2e58c00640 1 -- 172.21.15.7:0/3670237947 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2e4c018a90 con 0x7f2e5c109140 2024-04-20T21:35:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.968+0000 7f2e65637740 1 Processor -- start 2024-04-20T21:35:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.969+0000 7f2e65637740 1 -- start start 2024-04-20T21:35:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 0x55cc2679b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 0x55cc2679da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc266f8150 con 0x55cc267921a0 2024-04-20T21:35:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc2669e370 con 0x55cc2679b6a0 2024-04-20T21:35:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.970+0000 7f2e65637740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cc266f4110 con 0x55cc26792ac0 2024-04-20T21:35:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.971+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.971+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38058/0 (socket says 172.21.15.7:38058) 2024-04-20T21:35:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.971+0000 7f2e60a00640 1 -- 172.21.15.7:0/78538906 learned_addr learned my addr 172.21.15.7:0/78538906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.972+0000 7f2e60a00640 1 -- 172.21.15.7:0/78538906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 msgr2=0x55cc2679b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e5b400640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 0x55cc2679da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e5be00640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 0x55cc2679b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e60a00640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 0x55cc2679b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e60a00640 1 -- 172.21.15.7:0/78538906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 msgr2=0x55cc2679da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e60a00640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 0x55cc2679da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e60a00640 1 -- 172.21.15.7:0/78538906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc264d47d0 con 0x55cc267921a0 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e5b400640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 0x55cc2679da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.973+0000 7f2e5be00640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 0x55cc2679b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.974+0000 7f2e60a00640 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009e60 tx=0x7f2e5400f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee153cc9b05361f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.976+0000 7f2e51200640 1 -- 172.21.15.7:0/78538906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54003970 con 0x55cc267921a0 2024-04-20T21:35:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.977+0000 7f2e51200640 1 -- 172.21.15.7:0/78538906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54004030 con 0x55cc267921a0 2024-04-20T21:35:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.977+0000 7f2e51200640 1 -- 172.21.15.7:0/78538906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400cac0 con 0x55cc267921a0 2024-04-20T21:35:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e58c00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 0x7f2e400a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 -- 172.21.15.7:0/78538906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 msgr2=0x55cc26792580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009e60 tx=0x7f2e5400f7d0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e58c00640 1 -- 172.21.15.7:0/3670237947 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e4c0d11b0 con 0x7f2e5c109140 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 -- 172.21.15.7:0/78538906 shutdown_connections 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc2679b6a0 0x55cc2679da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc26792ac0 0x55cc2679b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 --2- 172.21.15.7:0/78538906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc26792580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.978+0000 7f2e65637740 1 -- 172.21.15.7:0/78538906 >> 172.21.15.7:0/78538906 conn(0x55cc2678c8d0 msgr2=0x55cc2678ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.982+0000 7f2e5be00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 0x7f2e400a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.982+0000 7f2e65637740 1 -- 172.21.15.7:0/78538906 shutdown_connections 2024-04-20T21:35:40.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.985+0000 7f2e5be00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 0x7f2e400a6240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0cea00 tx=0x7f2e4c0095b0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:40.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.985+0000 7f2e65637740 1 -- 172.21.15.7:0/78538906 wait complete. 2024-04-20T21:35:40.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.986+0000 7f2e65637740 1 -- start start 2024-04-20T21:35:40.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.986+0000 7f2e65637740 1 Processor -- start 2024-04-20T21:35:40.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc267986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26799140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cc2679e430 con 0x55cc2679b6a0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cc266b1740 con 0x55cc26792ac0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cc2679e5b0 con 0x55cc267921a0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e65637740 10 client.0 _ll_register_callbacks cb 0x55cc266acb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44500/0 (socket says 172.21.15.7:44500) 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e5b400640 1 -- 172.21.15.7:0/3853661387 learned_addr learned my addr 172.21.15.7:0/3853661387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.987+0000 7f2e60a00640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc267986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 -- 172.21.15.7:0/3853661387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 msgr2=0x55cc267986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc267986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 -- 172.21.15.7:0/3853661387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 msgr2=0x55cc26799140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26799140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 -- 172.21.15.7:0/3853661387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc2640a3f0 con 0x55cc26792ac0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e5b400640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009250 tx=0x7f2e48035f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9767e77f6f819034 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e45600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48039050 con 0x55cc26792ac0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e54004030 con 0x55cc26792ac0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.988+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e48004040 con 0x55cc26792ac0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.989+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e48004240 con 0x55cc26792ac0 2024-04-20T21:35:40.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.989+0000 7f2e65637740 10 client.25799 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.989+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc263ca0f0 con 0x55cc26792ac0 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.989+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54004030 con 0x55cc26792ac0 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.989+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f2e4802c950 con 0x55cc26792ac0 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 10 client.25799 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 20 client.25799 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc267ab900 0x55cc267add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 4 client.25799 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc26705b70 con 0x55cc267ab900 2024-04-20T21:35:40.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.990+0000 7f2e60a00640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc267ab900 0x55cc267add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:40.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.991+0000 7f2e45600640 10 client.25799 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:40.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.991+0000 7f2e60a00640 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc267ab900 0x55cc267add40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5586778d880d55ba server_cookie=a588fc52f2ba1ffb in_seq=0 out_seq=0 2024-04-20T21:35:40.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.992+0000 7f2e45600640 1 -- 172.21.15.7:0/3853661387 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc26705b70 con 0x55cc267ab900 2024-04-20T21:35:40.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.992+0000 7f2e45600640 10 client.25799 handle_command_reply: tid=0 2024-04-20T21:35:40.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 client.25799 shutdown 2024-04-20T21:35:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 msgr2=0x55cc26798c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009250 tx=0x7f2e48035f40 comp rx=0 tx=0).stop 2024-04-20T21:35:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 shutdown_connections 2024-04-20T21:35:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc26798c00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc267ab900 0x55cc267add40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26799140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 --2- 172.21.15.7:0/3853661387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc267986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e45600640 0 client.25799 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.993+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 >> 172.21.15.7:0/3853661387 conn(0x55cc2678c8d0 msgr2=0x55cc2678e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.994+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 shutdown_connections 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.994+0000 7f2e65637740 1 -- 172.21.15.7:0/3853661387 wait complete. 2024-04-20T21:35:40.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:40.994+0000 7f2e65637740 20 client.25799 trim_cache size 0 max 16384 2024-04-20T21:35:41.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.045+0000 7f2e65637740 1 Processor -- start 2024-04-20T21:35:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.045+0000 7f2e65637740 1 -- start start 2024-04-20T21:35:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.046+0000 7f2e65637740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc2679adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.048+0000 7f2e65637740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.048+0000 7f2e65637740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc2679b6a0 0x55cc2678ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.048+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc2679adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.049+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc2679adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38076/0 (socket says 172.21.15.7:38076) 2024-04-20T21:35:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.049+0000 7f2e5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.049+0000 7f2e5b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44516/0 (socket says 172.21.15.7:44516) 2024-04-20T21:35:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.050+0000 7f2e65637740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1bf3d0 con 0x55cc267921a0 2024-04-20T21:35:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.050+0000 7f2e65637740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1bcc90 con 0x55cc26792ac0 2024-04-20T21:35:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.050+0000 7f2e65637740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1baae0 con 0x55cc2679b6a0 2024-04-20T21:35:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.050+0000 7f2e5b400640 1 -- 172.21.15.7:0/1905740922 learned_addr learned my addr 172.21.15.7:0/1905740922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.051+0000 7f2e5b400640 1 -- 172.21.15.7:0/1905740922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc2679b6a0 msgr2=0x55cc2678ccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:35:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.051+0000 7f2e5b400640 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc2679b6a0 0x55cc2678ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.051+0000 7f2e5b400640 1 -- 172.21.15.7:0/1905740922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 msgr2=0x55cc2679adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.051+0000 7f2e5b400640 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc2679adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.051+0000 7f2e5b400640 1 -- 172.21.15.7:0/1905740922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc26796c10 con 0x55cc26792ac0 2024-04-20T21:35:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.052+0000 7f2e5b400640 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e4802c0d0 tx=0x7f2e48005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668b42a1e75bda42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.053+0000 7f2e47e00640 1 -- 172.21.15.7:0/1905740922 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48039070 con 0x55cc26792ac0 2024-04-20T21:35:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.053+0000 7f2e47e00640 1 -- 172.21.15.7:0/1905740922 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48057070 con 0x55cc26792ac0 2024-04-20T21:35:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.054+0000 7f2e47e00640 1 -- 172.21.15.7:0/1905740922 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48039070 con 0x55cc26792ac0 2024-04-20T21:35:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.054+0000 7f2e65637740 1 -- 172.21.15.7:0/1905740922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 msgr2=0x55cc267a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.054+0000 7f2e65637740 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e4802c0d0 tx=0x7f2e48005fb0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.055+0000 7f2e65637740 1 -- 172.21.15.7:0/1905740922 shutdown_connections 2024-04-20T21:35:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.055+0000 7f2e65637740 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc267a72f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.055+0000 7f2e65637740 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc2679b6a0 0x55cc2678ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.055+0000 7f2e65637740 1 --2- 172.21.15.7:0/1905740922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc267921a0 0x55cc2679adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.055+0000 7f2e65637740 1 -- 172.21.15.7:0/1905740922 >> 172.21.15.7:0/1905740922 conn(0x55cc2678c8d0 msgr2=0x55cc2678b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.056+0000 7f2e65637740 1 -- 172.21.15.7:0/1905740922 shutdown_connections 2024-04-20T21:35:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.056+0000 7f2e65637740 1 -- 172.21.15.7:0/1905740922 wait complete. 2024-04-20T21:35:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.057+0000 7f2e65637740 1 -- start start 2024-04-20T21:35:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.057+0000 7f2e65637740 1 Processor -- start 2024-04-20T21:35:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.061+0000 7f2e65637740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.061+0000 7f2e65637740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc2679fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.061+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.061+0000 7f2e60a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55420/0 (socket says 172.21.15.7:55420) 2024-04-20T21:35:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e60a00640 1 -- 172.21.15.7:0/115942332 learned_addr learned my addr 172.21.15.7:0/115942332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26810ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1bb150 con 0x55cc2679b6a0 2024-04-20T21:35:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1bc5c0 con 0x55cc26792ac0 2024-04-20T21:35:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1bb820 con 0x55cc267921a0 2024-04-20T21:35:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.063+0000 7f2e65637740 10 client.0 _ll_register_callbacks cb 0x7f2e48037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.062+0000 7f2e5b400640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc2679fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e60a00640 1 -- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 msgr2=0x55cc2679fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e60a00640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc2679fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e60a00640 1 -- 172.21.15.7:0/115942332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 msgr2=0x55cc26810ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e60a00640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26810ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e60a00640 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc267a02f0 con 0x55cc267921a0 2024-04-20T21:35:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.064+0000 7f2e5b400640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc2679fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.065+0000 7f2e60a00640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009e30 tx=0x7f2e54005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7c3f45fc4de8b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.065+0000 7f2e45600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.065+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540044f0 con 0x55cc267921a0 2024-04-20T21:35:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.066+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e54003970 con 0x55cc267921a0 2024-04-20T21:35:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.066+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e48057070 con 0x55cc267921a0 2024-04-20T21:35:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.066+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e54003b70 con 0x55cc267921a0 2024-04-20T21:35:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.066+0000 7f2e65637740 10 client.25795 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.066+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc26812520 con 0x55cc267921a0 2024-04-20T21:35:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48057070 con 0x55cc267921a0 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f2e5402d020 con 0x55cc267921a0 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 10 client.25795 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 10 client.25795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 20 client.25795 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc26812ae0 0x55cc26814f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 4 client.25795 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.067+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cc266ff9e0 con 0x55cc26812ae0 2024-04-20T21:35:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.068+0000 7f2e5b400640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc26812ae0 0x55cc26814f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.070+0000 7f2e45600640 10 client.25795 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.070+0000 7f2e5b400640 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc26812ae0 0x55cc26814f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de51af9e10a27c5 server_cookie=59eda06017c2de97 in_seq=0 out_seq=0 2024-04-20T21:35:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.071+0000 7f2e45600640 1 -- 172.21.15.7:0/115942332 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cc266ff9e0 con 0x55cc26812ae0 2024-04-20T21:35:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.071+0000 7f2e45600640 10 client.25795 handle_command_reply: tid=0 2024-04-20T21:35:41.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.071+0000 7f2e65637740 1 client.25795 shutdown 2024-04-20T21:35:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.071+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 msgr2=0x55cc2679f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.071+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009e30 tx=0x7f2e54005eb0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 shutdown_connections 2024-04-20T21:35:41.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cc26792ac0 0x55cc2679fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cc26812ae0 0x55cc26814f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cc2679b6a0 0x55cc26810ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 --2- 172.21.15.7:0/115942332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cc267921a0 0x55cc2679f520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 >> 172.21.15.7:0/115942332 conn(0x55cc2678c8d0 msgr2=0x55cc26798660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e45600640 0 client.25795 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 shutdown_connections 2024-04-20T21:35:41.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.072+0000 7f2e65637740 1 -- 172.21.15.7:0/115942332 wait complete. 2024-04-20T21:35:41.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.073+0000 7f2e65637740 20 client.25795 trim_cache size 0 max 16384 2024-04-20T21:35:41.072 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:35:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:35:41.072 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:41.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.079+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 msgr2=0x7f2e400a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.079+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 0x7f2e400a6240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0cea00 tx=0x7f2e4c0095b0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.079+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 msgr2=0x7f2e5c1a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.079+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e9f0 tx=0x7f2e4c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 shutdown_connections 2024-04-20T21:35:41.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e400a3d80 0x7f2e400a6240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1150b0 0x7f2e5c1abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c109a60 0x7f2e5c1a7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 --2- 172.21.15.7:0/3670237947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c109140 0x7f2e5c1a76e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 >> 172.21.15.7:0/3670237947 conn(0x7f2e5c06d000 msgr2=0x7f2e5c072020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.091+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 shutdown_connections 2024-04-20T21:35:41.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.092+0000 7f2e62a00640 1 -- 172.21.15.7:0/3670237947 wait complete. 2024-04-20T21:35:41.149 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:35:41.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 -- 172.21.15.7:0/1746035218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074c50 msgr2=0x7fc0cc10e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1746035218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074c50 0x7fc0cc10e290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ca0 tx=0x7fc0bc02c190 comp rx=0 tx=0).stop 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 -- 172.21.15.7:0/1746035218 shutdown_connections 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1746035218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074c50 0x7fc0cc10e290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1746035218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1746035218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc074330 0x7fc0cc074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 -- 172.21.15.7:0/1746035218 >> 172.21.15.7:0/1746035218 conn(0x7fc0cc0fda40 msgr2=0x7fc0cc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 -- 172.21.15.7:0/1746035218 shutdown_connections 2024-04-20T21:35:41.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.380+0000 7fc0d1400640 1 -- 172.21.15.7:0/1746035218 wait complete. 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 Processor -- start 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 -- start start 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 0x7fc0cc1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc114530 con 0x7fc0cc10e7d0 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc1143b0 con 0x7fc0cc074330 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.381+0000 7fc0d1400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc1146b0 con 0x7fc0cc074c50 2024-04-20T21:35:41.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0ca800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 0x7fc0cc1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55430/0 (socket says 172.21.15.7:55430) 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 -- 172.21.15.7:0/1991003273 learned_addr learned my addr 172.21.15.7:0/1991003273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0cb200640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 -- 172.21.15.7:0/1991003273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 msgr2=0x7fc0cc1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 0x7fc0cc1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 -- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 msgr2=0x7fc0cc19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0c9e00640 1 -- 172.21.15.7:0/1991003273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc19b070 con 0x7fc0cc074c50 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0ca800640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 0x7fc0cc1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.382+0000 7fc0cb200640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0c9e00640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc005e30 tx=0x7fc0bc035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc039040 con 0x7fc0cc074c50 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc19b300 con 0x7fc0cc074c50 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0bc044070 con 0x7fc0cc074c50 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0328d0 con 0x7fc0cc074c50 2024-04-20T21:35:41.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.383+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc1a7600 con 0x7fc0cc074c50 2024-04-20T21:35:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.384+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc069260 con 0x7fc0cc074c50 2024-04-20T21:35:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.386+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc0bc032a70 con 0x7fc0cc074c50 2024-04-20T21:35:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.387+0000 7fc0c3400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 0x7fc0b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.387+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc0bc0f2ff0 con 0x7fc0cc074c50 2024-04-20T21:35:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.388+0000 7fc0ca800640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 0x7fc0b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.388+0000 7fc0ca800640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 0x7fc0b00a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc19ba30 tx=0x7fc0b4005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.390+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc0bc0bf8b0 con 0x7fc0cc074c50 2024-04-20T21:35:41.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.628+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0cc19be20 con 0x7fc0cc074c50 2024-04-20T21:35:41.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.629+0000 7fc0c3400640 1 -- 172.21.15.7:0/1991003273 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7fc0bc0bf1b0 con 0x7fc0cc074c50 2024-04-20T21:35:41.628 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:35:41.628 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:35:06.444983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24559,24575,24535]},"id":1}]} 2024-04-20T21:35:41.628 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-04-20T21:35:41.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 msgr2=0x7fc0b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 0x7fc0b00a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc19ba30 tx=0x7fc0b4005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 msgr2=0x7fc0cc1a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc005e30 tx=0x7fc0bc035d90 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 shutdown_connections 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc10e7d0 0x7fc0cc19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074c50 0x7fc0cc1a0c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc0b00a3d20 0x7fc0b00a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 --2- 172.21.15.7:0/1991003273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0cc074330 0x7fc0cc1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.631+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 >> 172.21.15.7:0/1991003273 conn(0x7fc0cc0fda40 msgr2=0x7fc0cc0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.632+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 shutdown_connections 2024-04-20T21:35:41.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.632+0000 7fc0d1400640 1 -- 172.21.15.7:0/1991003273 wait complete. 2024-04-20T21:35:41.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:35:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 -- 172.21.15.7:0/3831842492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec681070a0 msgr2=0x7fec6810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 --2- 172.21.15.7:0/3831842492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec681070a0 0x7fec6810ec70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec60009ca0 tx=0x7fec6002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 -- 172.21.15.7:0/3831842492 shutdown_connections 2024-04-20T21:35:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 --2- 172.21.15.7:0/3831842492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec681070a0 0x7fec6810ec70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 --2- 172.21.15.7:0/3831842492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec68102980 0x7fec681069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 --2- 172.21.15.7:0/3831842492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec68101fd0 0x7fec681023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 -- 172.21.15.7:0/3831842492 >> 172.21.15.7:0/3831842492 conn(0x7fec680fd8b0 msgr2=0x7fec680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 -- 172.21.15.7:0/3831842492 shutdown_connections 2024-04-20T21:35:41.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.844+0000 7fec6ea00640 1 -- 172.21.15.7:0/3831842492 wait complete. 2024-04-20T21:35:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.845+0000 7fec6ea00640 1 Processor -- start 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.845+0000 7fec6ea00640 1 -- start start 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec681070a0 0x7fec681a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6819a660 0x7fec6819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec68112140 con 0x7fec681070a0 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec68111fc0 con 0x7fec68101fd0 2024-04-20T21:35:41.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6ea00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec681122c0 con 0x7fec6819a660 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec681070a0 0x7fec681a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44570/0 (socket says 172.21.15.7:44570) 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 -- 172.21.15.7:0/1601717315 learned_addr learned my addr 172.21.15.7:0/1601717315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6e000640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6819a660 0x7fec6819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 -- 172.21.15.7:0/1601717315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6819a660 msgr2=0x7fec6819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6819a660 0x7fec6819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 -- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec681070a0 msgr2=0x7fec681a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec681070a0 0x7fec681a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.846+0000 7fec6d600640 1 -- 172.21.15.7:0/1601717315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6819b260 con 0x7fec68101fd0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec6d600640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d950 tx=0x7fec5c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec66000640 1 -- 172.21.15.7:0/1601717315 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c017030 con 0x7fec68101fd0 2024-04-20T21:35:41.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec66000640 1 -- 172.21.15.7:0/1601717315 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec5c004020 con 0x7fec68101fd0 2024-04-20T21:35:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec66000640 1 -- 172.21.15.7:0/1601717315 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0047a0 con 0x7fec68101fd0 2024-04-20T21:35:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec681a7340 con 0x7fec68101fd0 2024-04-20T21:35:41.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.847+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec681a76f0 con 0x7fec68101fd0 2024-04-20T21:35:41.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.850+0000 7fec72dcc740 1 Processor -- start 2024-04-20T21:35:41.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.850+0000 7fec66000640 1 -- 172.21.15.7:0/1601717315 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fec5c0041d0 con 0x7fec68101fd0 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.850+0000 7fec72dcc740 1 -- start start 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.850+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61590550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec72dcc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bd614f5050 con 0x55bd61599670 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec72dcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd6149b490 con 0x55bd61590a90 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec72dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bd614f1010 con 0x55bd61590170 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61590550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44584/0 (socket says 172.21.15.7:44584) 2024-04-20T21:35:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6d600640 1 -- 172.21.15.7:0/2781433291 learned_addr learned my addr 172.21.15.7:0/2781433291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 -- 172.21.15.7:0/2781433291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 msgr2=0x55bd61599130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 -- 172.21.15.7:0/2781433291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 msgr2=0x55bd6159ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6e000640 1 -- 172.21.15.7:0/2781433291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd614a3070 con 0x55bd61590170 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.851+0000 7fec6d600640 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec6e000640 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61590550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec6002c710 tx=0x7fec60009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14023b9ced26e88f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec66000640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 0x7fec500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec5a600640 1 -- 172.21.15.7:0/2781433291 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60043070 con 0x55bd61590170 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec66000640 1 -- 172.21.15.7:0/1601717315 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fec5c0d0110 con 0x7fec68101fd0 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec5a600640 1 -- 172.21.15.7:0/2781433291 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec60003970 con 0x55bd61590170 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.852+0000 7fec5a600640 1 -- 172.21.15.7:0/2781433291 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60039040 con 0x55bd61590170 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec6d600640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 0x7fec500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 -- 172.21.15.7:0/2781433291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 msgr2=0x55bd61590550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61590550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec6002c710 tx=0x7fec60009980 comp rx=0 tx=0).stop 2024-04-20T21:35:41.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 -- 172.21.15.7:0/2781433291 shutdown_connections 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61599130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2781433291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61590550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 -- 172.21.15.7:0/2781433291 >> 172.21.15.7:0/2781433291 conn(0x55bd6158a8a0 msgr2=0x55bd6158ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec6d600640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 0x7fec500a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009610 tx=0x7fec5c0d3bc0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 -- 172.21.15.7:0/2781433291 shutdown_connections 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.853+0000 7fec72dcc740 1 -- 172.21.15.7:0/2781433291 wait complete. 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 -- start start 2024-04-20T21:35:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 Processor -- start 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61596c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd61597160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bd6159c400 con 0x55bd61599670 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd614ae860 con 0x55bd61590a90 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.854+0000 7fec72dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bd6159c580 con 0x55bd61590170 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec72dcc740 10 client.0 _ll_register_callbacks cb 0x55bd6147cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55480/0 (socket says 172.21.15.7:55480) 2024-04-20T21:35:41.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 -- 172.21.15.7:0/2774563913 learned_addr learned my addr 172.21.15.7:0/2774563913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd61597160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6cc00640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61596c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 -- 172.21.15.7:0/2774563913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 msgr2=0x55bd61596c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61596c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 -- 172.21.15.7:0/2774563913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 msgr2=0x55bd61597160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd61597160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 -- 172.21.15.7:0/2774563913 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd614b2950 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec6e000640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec6002c640 tx=0x7fec6002e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e4c92db115cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.855+0000 7fec4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60039070 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec60003970 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fec600327a0 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec6005b050 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec72dcc740 10 client.25813 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd61480820 con 0x55bd61590170 2024-04-20T21:35:41.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.856+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec60003970 con 0x55bd61590170 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fec60048790 con 0x55bd61590170 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 10 client.25813 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 10 client.25813 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 20 client.25813 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd615a94e0 0x55bd615ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 4 client.25813 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd61502a70 con 0x55bd615a94e0 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.857+0000 7fec6cc00640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd615a94e0 0x55bd615ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.858+0000 7fec6cc00640 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd615a94e0 0x55bd615ab8d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41877ae0450b6e90 server_cookie=1600de234decb8fc in_seq=0 out_seq=0 2024-04-20T21:35:41.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.858+0000 7fec4ea00640 10 client.25813 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:41.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec4ea00640 1 -- 172.21.15.7:0/2774563913 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd61502a70 con 0x55bd615a94e0 2024-04-20T21:35:41.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec4ea00640 10 client.25813 handle_command_reply: tid=0 2024-04-20T21:35:41.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 client.25813 shutdown 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 msgr2=0x55bd615966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec6002c640 tx=0x7fec6002e040 comp rx=0 tx=0).stop 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 shutdown_connections 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd615a94e0 0x55bd615ab8d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590a90 0x55bd61596c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd61597160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec4ea00640 0 client.25813 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 --2- 172.21.15.7:0/2774563913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd615966e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 >> 172.21.15.7:0/2774563913 conn(0x55bd6158a8a0 msgr2=0x55bd6158c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 shutdown_connections 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 1 -- 172.21.15.7:0/2774563913 wait complete. 2024-04-20T21:35:41.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.859+0000 7fec72dcc740 20 client.25813 trim_cache size 0 max 16384 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 Processor -- start 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 -- start start 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61590a90 0x55bd615a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61599670 0x55bd615b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec681b24c0 con 0x55bd61590a90 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec681afd80 con 0x55bd61599670 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.874+0000 7fec72dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec681adbd0 con 0x55bd61590170 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61590a90 0x55bd615a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55486/0 (socket says 172.21.15.7:55486) 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 -- 172.21.15.7:0/260744451 learned_addr learned my addr 172.21.15.7:0/260744451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61599670 0x55bd615b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 -- 172.21.15.7:0/260744451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61599670 msgr2=0x55bd615b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61599670 0x55bd615b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 -- 172.21.15.7:0/260744451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61590a90 msgr2=0x55bd615a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61590a90 0x55bd615a52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 -- 172.21.15.7:0/260744451 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd615abc40 con 0x55bd61590170 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec6e000640 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec60002230 tx=0x7fec6002c100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26edad0ce07fe3a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec59c00640 1 -- 172.21.15.7:0/260744451 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60039040 con 0x55bd61590170 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec59c00640 1 -- 172.21.15.7:0/260744451 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec60003970 con 0x55bd61590170 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.875+0000 7fec59c00640 1 -- 172.21.15.7:0/260744451 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60043040 con 0x55bd61590170 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 -- 172.21.15.7:0/260744451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 msgr2=0x55bd61598e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fec60002230 tx=0x7fec6002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 -- 172.21.15.7:0/260744451 shutdown_connections 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61599670 0x55bd615b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61590a90 0x55bd615a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 --2- 172.21.15.7:0/260744451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590170 0x55bd61598e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 -- 172.21.15.7:0/260744451 >> 172.21.15.7:0/260744451 conn(0x55bd6158a8a0 msgr2=0x55bd61595710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 -- 172.21.15.7:0/260744451 shutdown_connections 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.876+0000 7fec72dcc740 1 -- 172.21.15.7:0/260744451 wait complete. 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.877+0000 7fec72dcc740 1 -- start start 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.877+0000 7fec72dcc740 1 Processor -- start 2024-04-20T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 0x55bd615a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 0x55bd6159ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ae240 con 0x55bd61599670 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec681af6b0 con 0x55bd61590170 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fec681ae910 con 0x55bd61590a90 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec72dcc740 10 client.0 _ll_register_callbacks cb 0x7fec60032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 0x55bd615a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 0x55bd6159ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 0x55bd615a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44608/0 (socket says 172.21.15.7:44608) 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6e000640 1 -- 172.21.15.7:0/853005007 learned_addr learned my addr 172.21.15.7:0/853005007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6e000640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 -- 172.21.15.7:0/853005007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 msgr2=0x55bd615a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 0x55bd615a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 -- 172.21.15.7:0/853005007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 msgr2=0x55bd6159d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6cc00640 1 -- 172.21.15.7:0/853005007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd612073f0 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.878+0000 7fec6e000640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec6cc00640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 0x55bd6159ccf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55bd615abe70 tx=0x7fec5400bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e2d1b232e1f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec540202e0 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fec54020480 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec60003970 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fec54030050 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec72dcc740 10 client.25819 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.879+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd612d17d0 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec60003970 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fec54011bf0 con 0x55bd61590a90 2024-04-20T21:35:41.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 10 client.25819 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 10 client.25819 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 20 client.25819 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd6160f940 0x55bd61611d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 4 client.25819 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd614fc8e0 con 0x55bd6160f940 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.880+0000 7fec6e000640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd6160f940 0x55bd61611d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec6e000640 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd6160f940 0x55bd61611d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5237397e068a6df server_cookie=9858b60e6ed07381 in_seq=0 out_seq=0 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec4ea00640 10 client.25819 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec4ea00640 1 -- 172.21.15.7:0/853005007 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd614fc8e0 con 0x55bd6160f940 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec4ea00640 10 client.25819 handle_command_reply: tid=0 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 client.25819 shutdown 2024-04-20T21:35:41.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 msgr2=0x55bd6159ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 0x55bd6159ccf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55bd615abe70 tx=0x7fec5400bf50 comp rx=0 tx=0).stop 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 shutdown_connections 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bd61599670 0x55bd6159d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bd61590a90 0x55bd6159ccf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55bd6160f940 0x55bd61611d60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 --2- 172.21.15.7:0/853005007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd61590170 0x55bd615a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 >> 172.21.15.7:0/853005007 conn(0x55bd6158a8a0 msgr2=0x55bd61594e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec4ea00640 0 client.25819 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.881+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 shutdown_connections 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec72dcc740 1 -- 172.21.15.7:0/853005007 wait complete. 2024-04-20T21:35:41.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec72dcc740 20 client.25819 trim_cache size 0 max 16384 2024-04-20T21:35:41.888 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 msgr2=0x7fec500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 0x7fec500a6260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009610 tx=0x7fec5c0d3bc0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 msgr2=0x7fec681a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec5c00d950 tx=0x7fec5c00de10 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 shutdown_connections 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fec6819a660 0x7fec6819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec681070a0 0x7fec681a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fec500a3da0 0x7fec500a6260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 --2- 172.21.15.7:0/1601717315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec68101fd0 0x7fec681a05a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.882+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 >> 172.21.15.7:0/1601717315 conn(0x7fec680fd8b0 msgr2=0x7fec680ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.883+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 shutdown_connections 2024-04-20T21:35:41.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:41.883+0000 7fec6ea00640 1 -- 172.21.15.7:0/1601717315 wait complete. 2024-04-20T21:35:41.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:35:42.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 -- 172.21.15.7:0/3893205186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c109470 msgr2=0x7f660c111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 --2- 172.21.15.7:0/3893205186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c109470 0x7f660c111040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6604009ca0 tx=0x7f660402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 -- 172.21.15.7:0/3893205186 shutdown_connections 2024-04-20T21:35:42.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 --2- 172.21.15.7:0/3893205186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c104d50 0x7f660c108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 --2- 172.21.15.7:0/3893205186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c109470 0x7f660c111040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 --2- 172.21.15.7:0/3893205186 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 0x7f660c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 -- 172.21.15.7:0/3893205186 >> 172.21.15.7:0/3893205186 conn(0x7f660c0ffbe0 msgr2=0x7f660c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 -- 172.21.15.7:0/3893205186 shutdown_connections 2024-04-20T21:35:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.106+0000 7f6612600640 1 -- 172.21.15.7:0/3893205186 wait complete. 2024-04-20T21:35:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.107+0000 7f6612600640 1 Processor -- start 2024-04-20T21:35:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.107+0000 7f6612600640 1 -- start start 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 0x7f660c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 0x7f660c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f660c114530 con 0x7f660c19a850 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1143b0 con 0x7f660c109470 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f6612600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1146b0 con 0x7f660c1043a0 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f660be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f660b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 0x7f660c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f660be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38170/0 (socket says 172.21.15.7:38170) 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f660be00640 1 -- 172.21.15.7:0/1639627002 learned_addr learned my addr 172.21.15.7:0/1639627002 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.108+0000 7f660aa00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 0x7f660c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 -- 172.21.15.7:0/1639627002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 msgr2=0x7f660c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 0x7f660c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 -- 172.21.15.7:0/1639627002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 msgr2=0x7f660c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 0x7f660c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 -- 172.21.15.7:0/1639627002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c19b3e0 con 0x7f660c19a850 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660aa00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 0x7f660c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.109+0000 7f660be00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f660402c700 tx=0x7f6604032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.110+0000 7f6603e00640 1 -- 172.21.15.7:0/1639627002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604041020 con 0x7f660c19a850 2024-04-20T21:35:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.110+0000 7f6603e00640 1 -- 172.21.15.7:0/1639627002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6604034cd0 con 0x7f660c19a850 2024-04-20T21:35:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.110+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c19b670 con 0x7f660c19a850 2024-04-20T21:35:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.110+0000 7f6603e00640 1 -- 172.21.15.7:0/1639627002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604039040 con 0x7f660c19a850 2024-04-20T21:35:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.110+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c1a7820 con 0x7f660c19a850 2024-04-20T21:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.114+0000 7f66153fc740 1 Processor -- start 2024-04-20T21:35:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 -- start start 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b16b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b0d0050 con 0x55ee8b16afb0 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b076490 con 0x55ee8b16b8d0 2024-04-20T21:35:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.115+0000 7f66153fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b0cc010 con 0x55ee8b1744b0 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44638/0 (socket says 172.21.15.7:44638) 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 -- 172.21.15.7:0/574126095 learned_addr learned my addr 172.21.15.7:0/574126095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 -- 172.21.15.7:0/574126095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 msgr2=0x55ee8b1768a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 -- 172.21.15.7:0/574126095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 msgr2=0x55ee8b16b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b16b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.116+0000 7f660b400640 1 -- 172.21.15.7:0/574126095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8b07e070 con 0x55ee8b16b8d0 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.117+0000 7f660b400640 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009c90 tx=0x7f65fc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3034d46e77faa830 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.117+0000 7f6600c00640 1 -- 172.21.15.7:0/574126095 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc012450 con 0x55ee8b16b8d0 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.117+0000 7f6600c00640 1 -- 172.21.15.7:0/574126095 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65fc012ad0 con 0x55ee8b16b8d0 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.117+0000 7f6600c00640 1 -- 172.21.15.7:0/574126095 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00fc30 con 0x55ee8b16b8d0 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.118+0000 7f66153fc740 1 -- 172.21.15.7:0/574126095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 msgr2=0x55ee8b173f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.118+0000 7f66153fc740 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009c90 tx=0x7f65fc0117b0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f6603e00640 1 -- 172.21.15.7:0/1639627002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6604044020 con 0x7f660c19a850 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 -- 172.21.15.7:0/574126095 shutdown_connections 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b173f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 --2- 172.21.15.7:0/574126095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b16b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 -- 172.21.15.7:0/574126095 >> 172.21.15.7:0/574126095 conn(0x55ee8b1656e0 msgr2=0x55ee8b165ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 -- 172.21.15.7:0/574126095 shutdown_connections 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.120+0000 7f66153fc740 1 -- 172.21.15.7:0/574126095 wait complete. 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.121+0000 7f66153fc740 1 -- start start 2024-04-20T21:35:42.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 Processor -- start 2024-04-20T21:35:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b1714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b1744b0 0x55ee8b171f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b1772a0 con 0x55ee8b16afb0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b089860 con 0x55ee8b1744b0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ee8b177420 con 0x55ee8b16b8d0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.122+0000 7f66153fc740 10 client.0 _ll_register_callbacks cb 0x55ee8b057fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55540/0 (socket says 172.21.15.7:55540) 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 -- 172.21.15.7:0/3134079142 learned_addr learned my addr 172.21.15.7:0/3134079142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 -- 172.21.15.7:0/3134079142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b1744b0 msgr2=0x55ee8b171f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b1744b0 0x55ee8b171f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 -- 172.21.15.7:0/3134079142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 msgr2=0x55ee8b1714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b1714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.123+0000 7f660b400640 1 -- 172.21.15.7:0/3134079142 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8b08d950 con 0x55ee8b16b8d0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.124+0000 7f660b400640 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00fb90 tx=0x7f65fc00fbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d580d6f0a561c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.124+0000 7f65f5600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:35:42.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.124+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc029020 con 0x55ee8b16b8d0 2024-04-20T21:35:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.124+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65fc012ad0 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.124+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f65fc009d80 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.125+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65fc034050 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.125+0000 7f66153fc740 10 client.25828 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.125+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8b05b820 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.125+0000 7f6603e00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 0x7f65f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65fc012ad0 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f65fc0047f0 con 0x55ee8b16b8d0 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 10 client.25828 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 10 client.25828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 20 client.25828 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b18f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 4 client.25828 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee8b0dda70 con 0x55ee8b18cc50 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.126+0000 7f6603e00640 1 -- 172.21.15.7:0/1639627002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f66040f2450 con 0x7f660c19a850 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.127+0000 7f660aa00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 0x7f65f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.128+0000 7f660aa00640 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b18f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.128+0000 7f660aa00640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 0x7f65f00a60f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009fd0 tx=0x7f65f8009840 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.128+0000 7f660aa00640 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b18f040 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e277f83adad31ce server_cookie=c5abc50c097f74b8 in_seq=0 out_seq=0 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.128+0000 7f65f5600640 10 client.25828 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.129+0000 7f65f5600640 1 -- 172.21.15.7:0/3134079142 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee8b0dda70 con 0x55ee8b18cc50 2024-04-20T21:35:42.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.129+0000 7f65f5600640 10 client.25828 handle_command_reply: tid=0 2024-04-20T21:35:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.129+0000 7f66153fc740 1 client.25828 shutdown 2024-04-20T21:35:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 msgr2=0x55ee8b171a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00fb90 tx=0x7f65fc00fbc0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 shutdown_connections 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b18f040 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f65f5600640 0 client.25828 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b1744b0 0x55ee8b171f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b16b8d0 0x55ee8b171a20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 --2- 172.21.15.7:0/3134079142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b1714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.130+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 >> 172.21.15.7:0/3134079142 conn(0x55ee8b1656e0 msgr2=0x55ee8b167430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.131+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 shutdown_connections 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.131+0000 7f66153fc740 1 -- 172.21.15.7:0/3134079142 wait complete. 2024-04-20T21:35:42.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.131+0000 7f66153fc740 20 client.25828 trim_cache size 0 max 16384 2024-04-20T21:35:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.156+0000 7f66153fc740 1 Processor -- start 2024-04-20T21:35:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 -- start start 2024-04-20T21:35:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b173c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b1889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1ae380 con 0x55ee8b16afb0 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1abc70 con 0x55ee8b16b8d0 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.159+0000 7f66153fc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1a9ae0 con 0x55ee8b1744b0 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.160+0000 7f660be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.160+0000 7f660aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b1889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.160+0000 7f660be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:55544/0 (socket says 172.21.15.7:55544) 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.160+0000 7f660be00640 1 -- 172.21.15.7:0/2785648756 learned_addr learned my addr 172.21.15.7:0/2785648756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.161+0000 7f660be00640 1 -- 172.21.15.7:0/2785648756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 msgr2=0x55ee8b1889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.161+0000 7f660be00640 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b1889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.161+0000 7f660be00640 1 -- 172.21.15.7:0/2785648756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 msgr2=0x55ee8b173c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.161+0000 7f660be00640 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b173c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.161+0000 7f660be00640 1 -- 172.21.15.7:0/2785648756 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee8ade23f0 con 0x55ee8b1744b0 2024-04-20T21:35:42.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.162+0000 7f660be00640 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6604037040 tx=0x7f66040ba7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3bc6a229ef0fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.162+0000 7f65f7e00640 1 -- 172.21.15.7:0/2785648756 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66040bdeb0 con 0x55ee8b1744b0 2024-04-20T21:35:42.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.163+0000 7f65f7e00640 1 -- 172.21.15.7:0/2785648756 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6604034920 con 0x55ee8b1744b0 2024-04-20T21:35:42.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.163+0000 7f65f7e00640 1 -- 172.21.15.7:0/2785648756 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660404dd70 con 0x55ee8b1744b0 2024-04-20T21:35:42.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 -- 172.21.15.7:0/2785648756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 msgr2=0x55ee8b170640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6604037040 tx=0x7f66040ba7f0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 -- 172.21.15.7:0/2785648756 shutdown_connections 2024-04-20T21:35:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b1889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b170640 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 --2- 172.21.15.7:0/2785648756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b173c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 -- 172.21.15.7:0/2785648756 >> 172.21.15.7:0/2785648756 conn(0x55ee8b1656e0 msgr2=0x55ee8b163eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.164+0000 7f66153fc740 1 -- 172.21.15.7:0/2785648756 shutdown_connections 2024-04-20T21:35:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.165+0000 7f66153fc740 1 -- 172.21.15.7:0/2785648756 wait complete. 2024-04-20T21:35:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.165+0000 7f66153fc740 1 -- start start 2024-04-20T21:35:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.166+0000 7f66153fc740 1 Processor -- start 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.166+0000 7f66153fc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f660b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f660b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38206/0 (socket says 172.21.15.7:38206) 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f660b400640 1 -- 172.21.15.7:0/1873954257 learned_addr learned my addr 172.21.15.7:0/1873954257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b17c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1aa130 con 0x55ee8b16afb0 2024-04-20T21:35:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1ab5a0 con 0x55ee8b16b8d0 2024-04-20T21:35:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1aa800 con 0x55ee8b1744b0 2024-04-20T21:35:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f66153fc740 10 client.0 _ll_register_callbacks cb 0x7f65fc0128a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.167+0000 7f660be00640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.168+0000 7f660b400640 1 -- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 msgr2=0x55ee8b1777b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.168+0000 7f660b400640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1777b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.168+0000 7f660b400640 1 -- 172.21.15.7:0/1873954257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 msgr2=0x55ee8b17c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.168+0000 7f660b400640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b17c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.168+0000 7f660b400640 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8aeac7d0 con 0x55ee8b16afb0 2024-04-20T21:35:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.169+0000 7f660b400640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00fb60 tx=0x7f65fc012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e160b26ea145d8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.169+0000 7f65f5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:35:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.169+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc03d060 con 0x55ee8b16afb0 2024-04-20T21:35:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.170+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f65fc012950 con 0x55ee8b16afb0 2024-04-20T21:35:42.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.172+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6604034920 con 0x55ee8b16afb0 2024-04-20T21:35:42.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.173+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65fc029040 con 0x55ee8b16afb0 2024-04-20T21:35:42.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.173+0000 7f66153fc740 10 client.15720 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:42.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.173+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ee8b184a40 con 0x55ee8b16afb0 2024-04-20T21:35:42.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.174+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6604034920 con 0x55ee8b16afb0 2024-04-20T21:35:42.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.174+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f65fc0055b0 con 0x55ee8b16afb0 2024-04-20T21:35:42.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 10 client.15720 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:42.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 10 client.15720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:35:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 20 client.15720 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b185180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 4 client.15720 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.175+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee8b0d78e0 con 0x55ee8b18cc50 2024-04-20T21:35:42.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.176+0000 7f660aa00640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b185180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.180+0000 7f65f5600640 10 client.15720 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.180+0000 7f660aa00640 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b185180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c03a8f6d8855d3d2 server_cookie=cb84f1b1dc1f54b4 in_seq=0 out_seq=0 2024-04-20T21:35:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.180+0000 7f65f5600640 1 -- 172.21.15.7:0/1873954257 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee8b0d78e0 con 0x55ee8b18cc50 2024-04-20T21:35:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.180+0000 7f65f5600640 10 client.15720 handle_command_reply: tid=0 2024-04-20T21:35:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.181+0000 7f66153fc740 1 client.15720 shutdown 2024-04-20T21:35:42.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.183+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 msgr2=0x55ee8b17c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.183+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00fb60 tx=0x7f65fc012760 comp rx=0 tx=0).stop 2024-04-20T21:35:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 shutdown_connections 2024-04-20T21:35:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55ee8b18cc50 0x55ee8b185180 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ee8b16b8d0 0x55ee8b17c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ee8b1744b0 0x55ee8b1777b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 --2- 172.21.15.7:0/1873954257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ee8b16afb0 0x55ee8b17c3b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 >> 172.21.15.7:0/1873954257 conn(0x55ee8b1656e0 msgr2=0x55ee8b171d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.184+0000 7f65f5600640 0 client.15720 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:35:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.187+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 shutdown_connections 2024-04-20T21:35:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.188+0000 7f66153fc740 1 -- 172.21.15.7:0/1873954257 wait complete. 2024-04-20T21:35:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.188+0000 7f66153fc740 20 client.15720 trim_cache size 0 max 16384 2024-04-20T21:35:42.188 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 msgr2=0x7f65f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 0x7f65f00a60f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009fd0 tx=0x7f65f8009840 comp rx=0 tx=0).stop 2024-04-20T21:35:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 msgr2=0x7f660c19ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f660402c700 tx=0x7f6604032f20 comp rx=0 tx=0).stop 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 shutdown_connections 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65f00a3c30 0x7f65f00a60f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f660c109470 0x7f660c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.191+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f660c19a850 0x7f660c19ad00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.192+0000 7f6612600640 1 --2- 172.21.15.7:0/1639627002 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f660c1043a0 0x7f660c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.192+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 >> 172.21.15.7:0/1639627002 conn(0x7f660c0ffbe0 msgr2=0x7f660c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.195+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 shutdown_connections 2024-04-20T21:35:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.195+0000 7f6612600640 1 -- 172.21.15.7:0/1639627002 wait complete. 2024-04-20T21:35:42.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 -- 172.21.15.7:0/2438271523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c074c30 msgr2=0x7f1f1c10e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 --2- 172.21.15.7:0/2438271523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c074c30 0x7f1f1c10e0e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009900 tx=0x7f1f0802bcc0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 -- 172.21.15.7:0/2438271523 shutdown_connections 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 --2- 172.21.15.7:0/2438271523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c074c30 0x7f1f1c10e0e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 --2- 172.21.15.7:0/2438271523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c10e620 0x7f1f1c110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 --2- 172.21.15.7:0/2438271523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 -- 172.21.15.7:0/2438271523 >> 172.21.15.7:0/2438271523 conn(0x7f1f1c0fd8b0 msgr2=0x7f1f1c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 -- 172.21.15.7:0/2438271523 shutdown_connections 2024-04-20T21:35:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.465+0000 7f1f22400640 1 -- 172.21.15.7:0/2438271523 wait complete. 2024-04-20T21:35:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.466+0000 7f1f22400640 1 Processor -- start 2024-04-20T21:35:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.466+0000 7f1f22400640 1 -- start start 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 0x7f1f1c102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 0x7f1f1c1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c114360 con 0x7f1f1c074c30 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1141e0 con 0x7f1f1c10e620 2024-04-20T21:35:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f22400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c1144e0 con 0x7f1f1c074310 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f21a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 0x7f1f1c1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f21a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 0x7f1f1c1027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44676/0 (socket says 172.21.15.7:44676) 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f21000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f21a00640 1 -- 172.21.15.7:0/3057438672 learned_addr learned my addr 172.21.15.7:0/3057438672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.467+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 0x7f1f1c102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f21000640 1 -- 172.21.15.7:0/3057438672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 msgr2=0x7f1f1c1027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f21000640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 0x7f1f1c1027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f21000640 1 -- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 msgr2=0x7f1f1c102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f21000640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 0x7f1f1c102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f21000640 1 -- 172.21.15.7:0/3057438672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1aaf30 con 0x7f1f1c074310 2024-04-20T21:35:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.468+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 0x7f1f1c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:42.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.469+0000 7f1f21000640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c101d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e9f0 tx=0x7f1f1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.469+0000 7f1f19600640 1 -- 172.21.15.7:0/3057438672 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000cd80 con 0x7f1f1c074310 2024-04-20T21:35:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.470+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c1ab220 con 0x7f1f1c074310 2024-04-20T21:35:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.470+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c1ab790 con 0x7f1f1c074310 2024-04-20T21:35:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.470+0000 7f1f19600640 1 -- 172.21.15.7:0/3057438672 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f100035f0 con 0x7f1f1c074310 2024-04-20T21:35:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.470+0000 7f1f19600640 1 -- 172.21.15.7:0/3057438672 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10010670 con 0x7f1f1c074310 2024-04-20T21:35:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.473+0000 7f1f26837740 1 Processor -- start 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 -- start start 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad9131200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9131740 0x55cad914b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 0x55cad914da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad9096050 con 0x55cad9130e20 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad903c490 con 0x55cad914b680 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.474+0000 7f1f26837740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad9092010 con 0x55cad9131740 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad9131200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 0x55cad914da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 0x55cad914da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44690/0 (socket says 172.21.15.7:44690) 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f1be00640 1 -- 172.21.15.7:0/3390138408 learned_addr learned my addr 172.21.15.7:0/3390138408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 -- 172.21.15.7:0/3390138408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9131740 msgr2=0x55cad914b140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:35:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9131740 0x55cad914b140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 -- 172.21.15.7:0/3390138408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 msgr2=0x55cad914da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 0x55cad914da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.475+0000 7f1f21a00640 1 -- 172.21.15.7:0/3390138408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad9044070 con 0x55cad9130e20 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.476+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad9131200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009f30 tx=0x7f1f140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2300c51083110f79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.476+0000 7f1f0e000640 1 -- 172.21.15.7:0/3390138408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14004b20 con 0x55cad9130e20 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.476+0000 7f1f0e000640 1 -- 172.21.15.7:0/3390138408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f14004cc0 con 0x55cad9130e20 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.476+0000 7f1f0e000640 1 -- 172.21.15.7:0/3390138408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14005440 con 0x55cad9130e20 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.476+0000 7f1f19600640 1 -- 172.21.15.7:0/3057438672 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1f10004020 con 0x7f1f1c074310 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 -- 172.21.15.7:0/3390138408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 msgr2=0x55cad9131200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad9131200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009f30 tx=0x7f1f140118d0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 -- 172.21.15.7:0/3390138408 shutdown_connections 2024-04-20T21:35:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad914b680 0x55cad914da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9131740 0x55cad914b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 --2- 172.21.15.7:0/3390138408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad9131200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 -- 172.21.15.7:0/3390138408 >> 172.21.15.7:0/3390138408 conn(0x55cad912b8a0 msgr2=0x55cad912bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 -- 172.21.15.7:0/3390138408 shutdown_connections 2024-04-20T21:35:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.477+0000 7f1f26837740 1 -- 172.21.15.7:0/3390138408 wait complete. 2024-04-20T21:35:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 -- start start 2024-04-20T21:35:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 Processor -- start 2024-04-20T21:35:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad913c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 0x55cad913cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad9137cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cad9132310 con 0x55cad9130e20 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cad904f860 con 0x55cad9131740 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cad9132490 con 0x55cad914b680 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.478+0000 7f1f26837740 10 client.0 _ll_register_callbacks cb 0x55cad901dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad913c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 0x55cad913cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad913c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38234/0 (socket says 172.21.15.7:38234) 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21a00640 1 -- 172.21.15.7:0/4236795952 learned_addr learned my addr 172.21.15.7:0/4236795952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad9137cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 -- 172.21.15.7:0/4236795952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 msgr2=0x55cad9137cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad9137cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 -- 172.21.15.7:0/4236795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 msgr2=0x55cad913c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad913c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.479+0000 7f1f21000640 1 -- 172.21.15.7:0/4236795952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad9053950 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f21000640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 0x55cad913cdf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f100ce090 tx=0x7f1f100d6f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88daa273705667cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f02a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100e0020 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1f100dc850 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f14004cc0 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f100eb050 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.480+0000 7f1f26837740 10 client.25853 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cad9021820 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f14004cc0 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f19600640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 0x7f1f040a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f1f10098f50 con 0x55cad9131740 2024-04-20T21:35:42.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 10 client.25853 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 10 client.25853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 20 client.25853 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad9147bf0 0x55cad914a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 4 client.25853 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.481+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad90a3a70 con 0x55cad9147bf0 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.482+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad9147bf0 0x55cad914a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.482+0000 7f1f19600640 1 -- 172.21.15.7:0/3057438672 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1f100d0950 con 0x7f1f1c074310 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.483+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 0x7f1f040a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.483+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad9147bf0 0x55cad914a010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe34b591f45624a server_cookie=f2fcaad588786d98 in_seq=0 out_seq=0 2024-04-20T21:35:42.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.483+0000 7f1f02a00640 10 client.25853 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:42.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.484+0000 7f1f02a00640 1 -- 172.21.15.7:0/4236795952 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cad90a3a70 con 0x55cad9147bf0 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.484+0000 7f1f02a00640 10 client.25853 handle_command_reply: tid=0 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.484+0000 7f1f26837740 1 client.25853 shutdown 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 msgr2=0x55cad913cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 0x55cad913cdf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f100ce090 tx=0x7f1f100d6f90 comp rx=0 tx=0).stop 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 shutdown_connections 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9131740 0x55cad913cdf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad9147bf0 0x55cad914a010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad9137cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 --2- 172.21.15.7:0/4236795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9130e20 0x55cad913c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 >> 172.21.15.7:0/4236795952 conn(0x55cad912b8a0 msgr2=0x55cad912d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.485+0000 7f1f02a00640 0 client.25853 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:42.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.486+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 0x7f1f040a6170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400fef0 tx=0x7f1f140050c0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.486+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 shutdown_connections 2024-04-20T21:35:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.486+0000 7f1f26837740 1 -- 172.21.15.7:0/4236795952 wait complete. 2024-04-20T21:35:42.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.486+0000 7f1f26837740 20 client.25853 trim_cache size 0 max 16384 2024-04-20T21:35:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.501+0000 7f1f26837740 1 Processor -- start 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.501+0000 7f1f26837740 1 -- start start 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad914afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 0x55cad91532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad91537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc010f20 con 0x55cad9131740 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc00e7e0 con 0x55cad9130e20 2024-04-20T21:35:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f26837740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc00c630 con 0x55cad914b680 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad914afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad914afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44712/0 (socket says 172.21.15.7:44712) 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f21000640 1 -- 172.21.15.7:0/3977012166 learned_addr learned my addr 172.21.15.7:0/3977012166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 0x55cad91532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.502+0000 7f1f21a00640 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad91537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 -- 172.21.15.7:0/3977012166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 msgr2=0x55cad91537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad91537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 -- 172.21.15.7:0/3977012166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 msgr2=0x55cad914afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad914afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 -- 172.21.15.7:0/3977012166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad8da83f0 con 0x55cad9131740 2024-04-20T21:35:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.503+0000 7f1f1be00640 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 0x55cad91532b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f0802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f154853202a66a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.504+0000 7f1f0d600640 1 -- 172.21.15.7:0/3977012166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08004bb0 con 0x55cad9131740 2024-04-20T21:35:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.504+0000 7f1f0d600640 1 -- 172.21.15.7:0/3977012166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0806ab50 con 0x55cad9131740 2024-04-20T21:35:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.504+0000 7f1f0d600640 1 -- 172.21.15.7:0/3977012166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0802cd30 con 0x55cad9131740 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 -- 172.21.15.7:0/3977012166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 msgr2=0x55cad91532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 0x55cad91532b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1f08000c00 tx=0x7f1f0802e040 comp rx=0 tx=0).stop 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 -- 172.21.15.7:0/3977012166 shutdown_connections 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad914b680 0x55cad91537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad9131740 0x55cad91532b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 --2- 172.21.15.7:0/3977012166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad914afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.505+0000 7f1f26837740 1 -- 172.21.15.7:0/3977012166 >> 172.21.15.7:0/3977012166 conn(0x55cad912b8a0 msgr2=0x55cad914a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.506+0000 7f1f26837740 1 -- 172.21.15.7:0/3977012166 shutdown_connections 2024-04-20T21:35:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.506+0000 7f1f26837740 1 -- 172.21.15.7:0/3977012166 wait complete. 2024-04-20T21:35:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 -- start start 2024-04-20T21:35:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 Processor -- start 2024-04-20T21:35:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 0x55cad91402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9140830 0x55cad9140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc00cca0 con 0x55cad914b680 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc00e110 con 0x55cad9130e20 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc00d370 con 0x55cad9140830 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.507+0000 7f1f26837740 10 client.0 _ll_register_callbacks cb 0x7f1f100d6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44728/0 (socket says 172.21.15.7:44728) 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 -- 172.21.15.7:0/4239802337 learned_addr learned my addr 172.21.15.7:0/4239802337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 -- 172.21.15.7:0/4239802337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9140830 msgr2=0x55cad9140cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9140830 0x55cad9140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 -- 172.21.15.7:0/4239802337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 msgr2=0x55cad91402f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 0x55cad91402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.508+0000 7f1f21000640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 0x55cad91402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f21000640 1 -- 172.21.15.7:0/4239802337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad8e727d0 con 0x55cad9130e20 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 0x55cad91402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f21000640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f100d68f0 tx=0x7f1f100d6c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f844e1e7059f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f02a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100e0070 con 0x55cad9130e20 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1f100d64a0 con 0x55cad9130e20 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f0806ab50 con 0x55cad9130e20 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f1009c6b0 con 0x55cad9130e20 2024-04-20T21:35:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f26837740 10 client.25859 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.509+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cad91b0a70 con 0x55cad9130e20 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0806ab50 con 0x55cad9130e20 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f1f10004310 con 0x55cad9130e20 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 10 client.25859 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 10 client.25859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 20 client.25859 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad91b1000 0x55cad91b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 4 client.25859 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cad909d8e0 con 0x55cad91b1000 2024-04-20T21:35:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.510+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad91b1000 0x55cad91b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f1be00640 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad91b1000 0x55cad91b3440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb222e5715c4840 server_cookie=b51f251ecb025ef0 in_seq=0 out_seq=0 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f02a00640 10 client.25859 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f02a00640 1 -- 172.21.15.7:0/4239802337 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cad909d8e0 con 0x55cad91b1000 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f02a00640 10 client.25859 handle_command_reply: tid=0 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f26837740 1 client.25859 shutdown 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 msgr2=0x55cad9143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.511+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f100d68f0 tx=0x7f1f100d6c00 comp rx=0 tx=0).stop 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 shutdown_connections 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55cad91b1000 0x55cad91b3440 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cad9140830 0x55cad9140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cad914b680 0x55cad91402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 --2- 172.21.15.7:0/4239802337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cad9130e20 0x55cad9143fc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 >> 172.21.15.7:0/4239802337 conn(0x55cad912b8a0 msgr2=0x55cad912a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f02a00640 0 client.25859 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 shutdown_connections 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 1 -- 172.21.15.7:0/4239802337 wait complete. 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.512+0000 7f1f26837740 20 client.25859 trim_cache size 0 max 16384 2024-04-20T21:35:42.513 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 msgr2=0x7f1f040a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 0x7f1f040a6170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400fef0 tx=0x7f1f140050c0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 msgr2=0x7f1f1c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c101d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000e9f0 tx=0x7f1f1000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 shutdown_connections 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f040a3cb0 0x7f1f040a6170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1c10e620 0x7f1f1c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1c074c30 0x7f1f1c102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 --2- 172.21.15.7:0/3057438672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f1c074310 0x7f1f1c101d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.513+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 >> 172.21.15.7:0/3057438672 conn(0x7f1f1c0fd8b0 msgr2=0x7f1f1c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.514+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 shutdown_connections 2024-04-20T21:35:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:42.514+0000 7f1f22400640 1 -- 172.21.15.7:0/3057438672 wait complete. 2024-04-20T21:35:42.531 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:42 smithi007 ceph-mon[16867]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:42.531 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:42 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1991003273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:42 smithi016 ceph-mon[21409]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:42 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1991003273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:42 smithi186 ceph-mon[22157]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 127 MiB/s wr, 72 op/s 2024-04-20T21:35:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:42 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1991003273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:35:43.535 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:35:43.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:35:43.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 -- 172.21.15.7:0/380101437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018109480 msgr2=0x7f3018111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 --2- 172.21.15.7:0/380101437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018109480 0x7f3018111050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3010009ca0 tx=0x7f301002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 -- 172.21.15.7:0/380101437 shutdown_connections 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 --2- 172.21.15.7:0/380101437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3018109480 0x7f3018111050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 --2- 172.21.15.7:0/380101437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018104d60 0x7f3018108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 --2- 172.21.15.7:0/380101437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f3018104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 -- 172.21.15.7:0/380101437 >> 172.21.15.7:0/380101437 conn(0x7f30180ffbf0 msgr2=0x7f3018102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 -- 172.21.15.7:0/380101437 shutdown_connections 2024-04-20T21:35:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.828+0000 7f301fe00640 1 -- 172.21.15.7:0/380101437 wait complete. 2024-04-20T21:35:43.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.829+0000 7f301fe00640 1 Processor -- start 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.829+0000 7f301fe00640 1 -- start start 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018109480 0x7f30181a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301819a850 0x7f301819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3018114530 con 0x7f3018109480 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30181143b0 con 0x7f301819a850 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30181146b0 con 0x7f30181043b0 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40344/0 (socket says 172.21.15.7:40344) 2024-04-20T21:35:43.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 -- 172.21.15.7:0/3770537867 learned_addr learned my addr 172.21.15.7:0/3770537867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:43.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301ca00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018109480 0x7f30181a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 -- 172.21.15.7:0/3770537867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301819a850 msgr2=0x7f301819ad00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301819a850 0x7f301819ad00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 -- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018109480 msgr2=0x7f30181a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018109480 0x7f30181a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.830+0000 7f301d400640 1 -- 172.21.15.7:0/3770537867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301819b3e0 con 0x7f30181043b0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.831+0000 7f301d400640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.831+0000 7f3016000640 1 -- 172.21.15.7:0/3770537867 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c00cd80 con 0x7f30181043b0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.831+0000 7f3016000640 1 -- 172.21.15.7:0/3770537867 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300c005500 con 0x7f30181043b0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.831+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301819b6d0 con 0x7f30181043b0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.831+0000 7f3016000640 1 -- 172.21.15.7:0/3770537867 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c010690 con 0x7f30181043b0 2024-04-20T21:35:43.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.832+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30181a7860 con 0x7f30181043b0 2024-04-20T21:35:43.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.835+0000 7f3022a08740 1 Processor -- start 2024-04-20T21:35:43.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.835+0000 7f3022a08740 1 -- start start 2024-04-20T21:35:43.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555f53133d20 con 0x555f531cf6d0 2024-04-20T21:35:43.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f530da3e0 con 0x555f531ced20 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3022a08740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f5312fce0 con 0x555f531e0c40 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56708/0 (socket says 172.21.15.7:56708) 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 -- 172.21.15.7:0/4230339495 learned_addr learned my addr 172.21.15.7:0/4230339495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301ca00640 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 -- 172.21.15.7:0/4230339495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 msgr2=0x555f531e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 -- 172.21.15.7:0/4230339495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 msgr2=0x555f531e0700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e0700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f301de00640 1 -- 172.21.15.7:0/4230339495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f530f9980 con 0x555f531ced20 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.836+0000 7f3016000640 1 -- 172.21.15.7:0/3770537867 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f300c004f50 con 0x7f30181043b0 2024-04-20T21:35:43.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.837+0000 7f301de00640 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3010005e30 tx=0x7f3010005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1578beb97e8065cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.837+0000 7f300a600640 1 -- 172.21.15.7:0/4230339495 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010041020 con 0x555f531ced20 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.837+0000 7f300a600640 1 -- 172.21.15.7:0/4230339495 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010032c90 con 0x555f531ced20 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.837+0000 7f300a600640 1 -- 172.21.15.7:0/4230339495 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010039040 con 0x555f531ced20 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 -- 172.21.15.7:0/4230339495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 msgr2=0x555f531cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3010005e30 tx=0x7f3010005fd0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 -- 172.21.15.7:0/4230339495 shutdown_connections 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 --2- 172.21.15.7:0/4230339495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f531cf100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3022a08740 1 -- 172.21.15.7:0/4230339495 >> 172.21.15.7:0/4230339495 conn(0x555f531c9580 msgr2=0x555f531c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3016000640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f3016000640 1 -- 172.21.15.7:0/3770537867 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f300c0d0ca0 con 0x7f30181043b0 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.838+0000 7f301ca00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.839+0000 7f3022a08740 1 -- 172.21.15.7:0/4230339495 shutdown_connections 2024-04-20T21:35:43.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.839+0000 7f3022a08740 1 -- 172.21.15.7:0/4230339495 wait complete. 2024-04-20T21:35:43.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.839+0000 7f301ca00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 0x7f30000a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555f531caa60 tx=0x7f3004005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.839+0000 7f3022a08740 1 -- start start 2024-04-20T21:35:43.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 Processor -- start 2024-04-20T21:35:43.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555f531e3970 con 0x555f531ced20 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f530ed7a0 con 0x555f531cf6d0 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555f531e3af0 con 0x555f531e0c40 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f3022a08740 10 client.0 _ll_register_callbacks cb 0x555f530efb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f301ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f301ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56710/0 (socket says 172.21.15.7:56710) 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f301ca00640 1 -- 172.21.15.7:0/1727872109 learned_addr learned my addr 172.21.15.7:0/1727872109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.840+0000 7f301d400640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 -- 172.21.15.7:0/1727872109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 msgr2=0x555f531de750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 -- 172.21.15.7:0/1727872109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 msgr2=0x555f531ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531ddcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 -- 172.21.15.7:0/1727872109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f52f0e7e0 con 0x555f531cf6d0 2024-04-20T21:35:43.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f301ca00640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f300401e610 tx=0x7f3004004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3f8203a9e60cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f2ffea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300402e060 con 0x555f531cf6d0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3004018030 con 0x555f531cf6d0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.841+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010032c90 con 0x555f531cf6d0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f300401ed80 con 0x555f531cf6d0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 10 client.25868 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f52f6ef70 con 0x555f531cf6d0 2024-04-20T21:35:43.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010032c90 con 0x555f531cf6d0 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f30040324f0 con 0x555f531cf6d0 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 10 client.25868 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 20 client.25868 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f531f0ff0 0x555f531f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 4 client.25868 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.842+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f53141740 con 0x555f531f0ff0 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.843+0000 7f301d400640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f531f0ff0 0x555f531f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.843+0000 7f2ffea00640 10 client.25868 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:43.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.845+0000 7f301d400640 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f531f0ff0 0x555f531f3430 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ea60443305f607 server_cookie=a93a419d3a6d0928 in_seq=0 out_seq=0 2024-04-20T21:35:43.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.846+0000 7f2ffea00640 1 -- 172.21.15.7:0/1727872109 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f53141740 con 0x555f531f0ff0 2024-04-20T21:35:43.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.846+0000 7f2ffea00640 10 client.25868 handle_command_reply: tid=0 2024-04-20T21:35:43.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 client.25868 shutdown 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 msgr2=0x555f531de210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f300401e610 tx=0x7f3004004730 comp rx=0 tx=0).stop 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 shutdown_connections 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531de210 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f531f0ff0 0x555f531f3430 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531de750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 --2- 172.21.15.7:0/1727872109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 >> 172.21.15.7:0/1727872109 conn(0x555f531c9580 msgr2=0x555f531cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f2ffea00640 0 client.25868 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.847+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 shutdown_connections 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.848+0000 7f3022a08740 1 -- 172.21.15.7:0/1727872109 wait complete. 2024-04-20T21:35:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.848+0000 7f3022a08740 20 client.25868 trim_cache size 0 max 16384 2024-04-20T21:35:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 Processor -- start 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 -- start start 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30181ae280 con 0x555f531ced20 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30181abb40 con 0x555f531cf6d0 2024-04-20T21:35:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.875+0000 7f3022a08740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30181a9990 con 0x555f531e0c40 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40386/0 (socket says 172.21.15.7:40386) 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 -- 172.21.15.7:0/4194791581 learned_addr learned my addr 172.21.15.7:0/4194791581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301ca00640 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 -- 172.21.15.7:0/4194791581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 msgr2=0x555f531ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 -- 172.21.15.7:0/4194791581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 msgr2=0x555f531e0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531e0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.876+0000 7f301de00640 1 -- 172.21.15.7:0/4194791581 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f52e06170 con 0x555f531e0c40 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.877+0000 7f301de00640 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3010009ca0 tx=0x7f301002c100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edcef55da5658b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.877+0000 7f3009c00640 1 -- 172.21.15.7:0/4194791581 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301004a020 con 0x555f531e0c40 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3009c00640 1 -- 172.21.15.7:0/4194791581 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010003970 con 0x555f531e0c40 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3009c00640 1 -- 172.21.15.7:0/4194791581 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010039070 con 0x555f531e0c40 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 -- 172.21.15.7:0/4194791581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 msgr2=0x555f531dce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3010009ca0 tx=0x7f301002c100 comp rx=0 tx=0).stop 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 -- 172.21.15.7:0/4194791581 shutdown_connections 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531cf6d0 0x555f531ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531dce30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 --2- 172.21.15.7:0/4194791581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531ced20 0x555f531e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.878+0000 7f3022a08740 1 -- 172.21.15.7:0/4194791581 >> 172.21.15.7:0/4194791581 conn(0x555f531c9580 msgr2=0x555f531caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.879+0000 7f3022a08740 1 -- 172.21.15.7:0/4194791581 shutdown_connections 2024-04-20T21:35:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.879+0000 7f3022a08740 1 -- 172.21.15.7:0/4194791581 wait complete. 2024-04-20T21:35:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.879+0000 7f3022a08740 1 -- start start 2024-04-20T21:35:43.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 Processor -- start 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f53252b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30181aa000 con 0x555f531cf6d0 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30181ab470 con 0x555f531ced20 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f30181aa6d0 con 0x555f531e0c40 2024-04-20T21:35:43.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.880+0000 7f3022a08740 10 client.0 _ll_register_callbacks cb 0x7f3004009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58630/0 (socket says 172.21.15.7:58630) 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301ca00640 1 -- 172.21.15.7:0/1650208016 learned_addr learned my addr 172.21.15.7:0/1650208016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f53252b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 -- 172.21.15.7:0/1650208016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 msgr2=0x555f531e92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 -- 172.21.15.7:0/1650208016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 msgr2=0x555f531e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 -- 172.21.15.7:0/1650208016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f531da900 con 0x555f531ced20 2024-04-20T21:35:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301de00640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f301d400640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f53252b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f300c0ce3e0 tx=0x7f300c0bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d964a76b58d2ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.881+0000 7f2ffea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c087040 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f300c005240 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010003970 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f300c09cc30 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f3022a08740 10 client.25874 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f532688a0 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.882+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010003970 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f3004009ce0 con 0x555f531ced20 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 10 client.25874 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 10 client.25874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 20 client.25874 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f53269000 0x555f5326b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 4 client.25874 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:35:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f5313b5b0 con 0x555f53269000 2024-04-20T21:35:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.883+0000 7f301ca00640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f53269000 0x555f5326b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.884+0000 7f2ffea00640 10 client.25874 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.887+0000 7f301ca00640 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f53269000 0x555f5326b420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fa8d6832ca73c6 server_cookie=206f5480d86cd336 in_seq=0 out_seq=0 2024-04-20T21:35:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.888+0000 7f2ffea00640 1 -- 172.21.15.7:0/1650208016 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f5313b5b0 con 0x555f53269000 2024-04-20T21:35:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.888+0000 7f2ffea00640 10 client.25874 handle_command_reply: tid=0 2024-04-20T21:35:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.889+0000 7f3022a08740 1 client.25874 shutdown 2024-04-20T21:35:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.889+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 msgr2=0x555f53252b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.889+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f53252b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f300c0ce3e0 tx=0x7f300c0bd040 comp rx=0 tx=0).stop 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 shutdown_connections 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x555f53269000 0x555f5326b420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555f531e0c40 0x555f531e92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555f531cf6d0 0x555f531e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 --2- 172.21.15.7:0/1650208016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f531ced20 0x555f53252b30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 >> 172.21.15.7:0/1650208016 conn(0x555f531c9580 msgr2=0x555f531cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f2ffea00640 0 client.25874 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 shutdown_connections 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.890+0000 7f3022a08740 1 -- 172.21.15.7:0/1650208016 wait complete. 2024-04-20T21:35:43.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.891+0000 7f3022a08740 20 client.25874 trim_cache size 0 max 16384 2024-04-20T21:35:43.895 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:35:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "03906d40-9b2a-4ca2-8c02-51de7a09f537", 2024-04-20T21:35:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:35:43.895 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.896+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 msgr2=0x7f30000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.896+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 0x7f30000a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555f531caa60 tx=0x7f3004005f90 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.896+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 msgr2=0x7f30181a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.896+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00edf0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 shutdown_connections 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f30000a3da0 0x7f30000a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f301819a850 0x7f301819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3018109480 0x7f30181a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 --2- 172.21.15.7:0/3770537867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30181043b0 0x7f30181a06e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.897+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 >> 172.21.15.7:0/3770537867 conn(0x7f30180ffbf0 msgr2=0x7f3018101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.903+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 shutdown_connections 2024-04-20T21:35:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:43.904+0000 7f301fe00640 1 -- 172.21.15.7:0/3770537867 wait complete. 2024-04-20T21:35:43.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:35:43.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 -- 172.21.15.7:0/2914479187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 msgr2=0x7f47c410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 --2- 172.21.15.7:0/2914479187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c410e9d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009be0 tx=0x7f47bc02bf80 comp rx=0 tx=0).stop 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 -- 172.21.15.7:0/2914479187 shutdown_connections 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 --2- 172.21.15.7:0/2914479187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c4103af0 0x7f47c410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 --2- 172.21.15.7:0/2914479187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c410e9d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.341+0000 7f47cc000640 1 --2- 172.21.15.7:0/2914479187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c41031d0 0x7f47c41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.342+0000 7f47cc000640 1 -- 172.21.15.7:0/2914479187 >> 172.21.15.7:0/2914479187 conn(0x7f47c4077f50 msgr2=0x7f47c4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.342+0000 7f47cc000640 1 -- 172.21.15.7:0/2914479187 shutdown_connections 2024-04-20T21:35:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.342+0000 7f47cc000640 1 -- 172.21.15.7:0/2914479187 wait complete. 2024-04-20T21:35:44.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.343+0000 7f47cc000640 1 Processor -- start 2024-04-20T21:35:44.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 -- start start 2024-04-20T21:35:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 0x7f47c41a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 0x7f47c41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4112360 con 0x7f47c4103af0 2024-04-20T21:35:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41121e0 con 0x7f47c41031d0 2024-04-20T21:35:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.344+0000 7f47cc000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41124e0 con 0x7f47c410c5a0 2024-04-20T21:35:44.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.346+0000 7f47c8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 0x7f47c41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.347+0000 7f47ca000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.347+0000 7f47ca000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40404/0 (socket says 172.21.15.7:40404) 2024-04-20T21:35:44.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.347+0000 7f47ca000640 1 -- 172.21.15.7:0/1840474088 learned_addr learned my addr 172.21.15.7:0/1840474088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:44.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.348+0000 7f47c9600640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 0x7f47c41a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.349+0000 7f47c9600640 1 -- 172.21.15.7:0/1840474088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 msgr2=0x7f47c419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c9600640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c9600640 1 -- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 msgr2=0x7f47c41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c9600640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 0x7f47c41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c9600640 1 -- 172.21.15.7:0/1840474088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c419b020 con 0x7f47c41031d0 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c8c00640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 0x7f47c41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47ca000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:44.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c9600640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 0x7f47c41a06e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47b0005fd0 tx=0x7f47b000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.351+0000 7f47c2000640 1 -- 172.21.15.7:0/1840474088 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000cd00 con 0x7f47c41031d0 2024-04-20T21:35:44.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.352+0000 7f47c2000640 1 -- 172.21.15.7:0/1840474088 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b00035f0 con 0x7f47c41031d0 2024-04-20T21:35:44.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.352+0000 7f47c2000640 1 -- 172.21.15.7:0/1840474088 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0010670 con 0x7f47c41031d0 2024-04-20T21:35:44.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.352+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c419b310 con 0x7f47c41031d0 2024-04-20T21:35:44.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.352+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a7530 con 0x7f47c41031d0 2024-04-20T21:35:44.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.357+0000 7f47ced68740 1 Processor -- start 2024-04-20T21:35:44.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.357+0000 7f47ced68740 1 -- start start 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 0x559cea894580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea894ac0 0x559cea89d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea89fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cea7fa150 con 0x559cea8941a0 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cea7a0370 con 0x559cea89d6a0 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ced68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559cea7f6110 con 0x559cea894ac0 2024-04-20T21:35:44.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 0x559cea894580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 0x559cea894580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58652/0 (socket says 172.21.15.7:58652) 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47ca000640 1 -- 172.21.15.7:0/647297369 learned_addr learned my addr 172.21.15.7:0/647297369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.358+0000 7f47c8c00640 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea89fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 -- 172.21.15.7:0/647297369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea894ac0 msgr2=0x559cea89d160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea894ac0 0x559cea89d160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 -- 172.21.15.7:0/647297369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 msgr2=0x559cea894580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 0x559cea894580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 -- 172.21.15.7:0/647297369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cea5d67d0 con 0x559cea89d6a0 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47c8c00640 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea89fa90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47c419bba0 tx=0x7f47b80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747cade07905e420 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.359+0000 7f47b6600640 1 -- 172.21.15.7:0/647297369 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004220 con 0x559cea89d6a0 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.360+0000 7f47b6600640 1 -- 172.21.15.7:0/647297369 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b80043c0 con 0x559cea89d6a0 2024-04-20T21:35:44.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.360+0000 7f47b6600640 1 -- 172.21.15.7:0/647297369 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b80035f0 con 0x559cea89d6a0 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 -- 172.21.15.7:0/647297369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 msgr2=0x559cea89fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea89fa90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47c419bba0 tx=0x7f47b80118d0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 -- 172.21.15.7:0/647297369 shutdown_connections 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea89fa90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea894ac0 0x559cea89d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 --2- 172.21.15.7:0/647297369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea8941a0 0x559cea894580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.361+0000 7f47ced68740 1 -- 172.21.15.7:0/647297369 >> 172.21.15.7:0/647297369 conn(0x559cea88e8d0 msgr2=0x559cea88ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.365+0000 7f47c2000640 1 -- 172.21.15.7:0/1840474088 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f47b00037a0 con 0x7f47c41031d0 2024-04-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.365+0000 7f47ced68740 1 -- 172.21.15.7:0/647297369 shutdown_connections 2024-04-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.365+0000 7f47ced68740 1 -- 172.21.15.7:0/647297369 wait complete. 2024-04-20T21:35:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.366+0000 7f47ced68740 1 -- start start 2024-04-20T21:35:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.366+0000 7f47ced68740 1 Processor -- start 2024-04-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 0x559cea89acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cea8a0490 con 0x559cea89b210 2024-04-20T21:35:44.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cea7b3740 con 0x559cea8941a0 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559cea8a0610 con 0x559cea89d6a0 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ced68740 10 client.0 _ll_register_callbacks cb 0x559cea7aeb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56758/0 (socket says 172.21.15.7:56758) 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47ca000640 1 -- 172.21.15.7:0/2655091077 learned_addr learned my addr 172.21.15.7:0/2655091077 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.367+0000 7f47c9600640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 0x559cea89acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47c2000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 0x7f47ac0a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 -- 172.21.15.7:0/2655091077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 msgr2=0x559cea89acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47c2000640 1 -- 172.21.15.7:0/1840474088 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f47b00d0950 con 0x7f47c41031d0 2024-04-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 0x559cea89acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 -- 172.21.15.7:0/2655091077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 msgr2=0x559cea8a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 -- 172.21.15.7:0/2655091077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cea50c3f0 con 0x559cea8941a0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47c9600640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 0x559cea89acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0334c0 tx=0x7f47bc003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e4a6420d2d1f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.368+0000 7f47ca000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 0x7f47ac0a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.369+0000 7f47aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.369+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc051060 con 0x559cea8941a0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.369+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f47bc004480 con 0x559cea8941a0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.370+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47b80043c0 con 0x559cea8941a0 2024-04-20T21:35:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.369+0000 7f47ca000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 0x7f47ac0a6170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47bc04e000 tx=0x7f47bc04b000 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.370+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47bc004860 con 0x559cea8941a0 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.370+0000 7f47ced68740 10 client.25886 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.370+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559cea4cc0f0 con 0x559cea8941a0 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.370+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b80043c0 con 0x559cea8941a0 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f47bc049040 con 0x559cea8941a0 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 10 client.25886 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 20 client.25886 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea8b6860 0x559cea8b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 4 client.25886 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cea807b70 con 0x559cea8b6860 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.371+0000 7f47c9600640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea8b6860 0x559cea8b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.372+0000 7f47c9600640 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea8b6860 0x559cea8b8ce0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1dae6bbcd91f71 server_cookie=1c3a2e1d6e6acba2 in_seq=0 out_seq=0 2024-04-20T21:35:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.372+0000 7f47aaa00640 10 client.25886 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.372+0000 7f47aaa00640 1 -- 172.21.15.7:0/2655091077 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cea807b70 con 0x559cea8b6860 2024-04-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.372+0000 7f47aaa00640 10 client.25886 handle_command_reply: tid=0 2024-04-20T21:35:44.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.373+0000 7f47ced68740 1 client.25886 shutdown 2024-04-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 msgr2=0x559cea89a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0334c0 tx=0x7f47bc003f50 comp rx=0 tx=0).stop 2024-04-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 shutdown_connections 2024-04-20T21:35:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea8b6860 0x559cea8b8ce0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea89d6a0 0x559cea89acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 --2- 172.21.15.7:0/2655091077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea8941a0 0x559cea89a790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 >> 172.21.15.7:0/2655091077 conn(0x559cea88e8d0 msgr2=0x559cea890620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.374+0000 7f47aaa00640 0 client.25886 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.377+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 shutdown_connections 2024-04-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.377+0000 7f47ced68740 1 -- 172.21.15.7:0/2655091077 wait complete. 2024-04-20T21:35:44.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.377+0000 7f47ced68740 20 client.25886 trim_cache size 0 max 16384 2024-04-20T21:35:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.410+0000 7f47ced68740 1 Processor -- start 2024-04-20T21:35:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.410+0000 7f47ced68740 1 -- start start 2024-04-20T21:35:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41ae3f0 con 0x559cea89b210 2024-04-20T21:35:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41abcb0 con 0x559cea89d6a0 2024-04-20T21:35:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.411+0000 7f47ced68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41a9b20 con 0x559cea8941a0 2024-04-20T21:35:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.412+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.412+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56764/0 (socket says 172.21.15.7:56764) 2024-04-20T21:35:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.412+0000 7f47ca000640 1 -- 172.21.15.7:0/820519647 learned_addr learned my addr 172.21.15.7:0/820519647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.412+0000 7f47c9600640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.412+0000 7f47c8c00640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47ca000640 1 -- 172.21.15.7:0/820519647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 msgr2=0x559cea8be5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47ca000640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8be5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47ca000640 1 -- 172.21.15.7:0/820519647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 msgr2=0x559cea8beaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47ca000640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8beaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47ca000640 1 -- 172.21.15.7:0/820519647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cea888d10 con 0x559cea89d6a0 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.414+0000 7f47c8c00640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8beaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:35:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.415+0000 7f47ca000640 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47bc036040 tx=0x7f47bc02bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43d58d03e31c7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.415+0000 7f47b5c00640 1 -- 172.21.15.7:0/820519647 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0043c0 con 0x559cea89d6a0 2024-04-20T21:35:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.415+0000 7f47b5c00640 1 -- 172.21.15.7:0/820519647 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc05d020 con 0x559cea89d6a0 2024-04-20T21:35:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.415+0000 7f47b5c00640 1 -- 172.21.15.7:0/820519647 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0043c0 con 0x559cea89d6a0 2024-04-20T21:35:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.417+0000 7f47ced68740 1 -- 172.21.15.7:0/820519647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 msgr2=0x559cea8a93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.417+0000 7f47ced68740 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f47bc036040 tx=0x7f47bc02bf80 comp rx=0 tx=0).stop 2024-04-20T21:35:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 -- 172.21.15.7:0/820519647 shutdown_connections 2024-04-20T21:35:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a93b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8beaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 --2- 172.21.15.7:0/820519647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 -- 172.21.15.7:0/820519647 >> 172.21.15.7:0/820519647 conn(0x559cea88e8d0 msgr2=0x559cea899060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.418+0000 7f47ced68740 1 -- 172.21.15.7:0/820519647 shutdown_connections 2024-04-20T21:35:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.419+0000 7f47ced68740 1 -- 172.21.15.7:0/820519647 wait complete. 2024-04-20T21:35:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.419+0000 7f47ced68740 1 -- start start 2024-04-20T21:35:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.420+0000 7f47ced68740 1 Processor -- start 2024-04-20T21:35:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.421+0000 7f47ced68740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.421+0000 7f47ced68740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.421+0000 7f47ced68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ced68740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41aa170 con 0x559cea89b210 2024-04-20T21:35:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ced68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41ab5e0 con 0x559cea89d6a0 2024-04-20T21:35:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ced68740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f47c41aa840 con 0x559cea8941a0 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ced68740 10 client.0 _ll_register_callbacks cb 0x7f47bc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56780/0 (socket says 172.21.15.7:56780) 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 -- 172.21.15.7:0/941332344 learned_addr learned my addr 172.21.15.7:0/941332344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47c9600640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 -- 172.21.15.7:0/941332344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 msgr2=0x559cea8a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 -- 172.21.15.7:0/941332344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 msgr2=0x559cea8a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.422+0000 7f47ca000640 1 -- 172.21.15.7:0/941332344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cea91bf80 con 0x559cea89d6a0 2024-04-20T21:35:44.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.423+0000 7f47ca000640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005d70 tx=0x7f47bc0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0cfede182df5a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:35:44.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.423+0000 7f47aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:35:44.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.423+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc068020 con 0x559cea89d6a0 2024-04-20T21:35:44.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.423+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f47bc04b870 con 0x559cea89d6a0 2024-04-20T21:35:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.424+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47bc05d020 con 0x559cea89d6a0 2024-04-20T21:35:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.424+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47bc02ccc0 con 0x559cea89d6a0 2024-04-20T21:35:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.424+0000 7f47ced68740 10 client.25892 fetch_fsmap learned FSMap version 25 2024-04-20T21:35:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.424+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559cea91d270 con 0x559cea89d6a0 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47bc05d020 con 0x559cea89d6a0 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f47bc049040 con 0x559cea89d6a0 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 10 client.25892 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 20 client.25892 populate_metadata read hostname 'smithi007' 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea91d9d0 0x559cea91fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:35:44.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 4 client.25892 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:35:44.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.425+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cea8019e0 con 0x559cea91d9d0 2024-04-20T21:35:44.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.426+0000 7f47c9600640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea91d9d0 0x559cea91fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:35:44.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.427+0000 7f47aaa00640 10 client.25892 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.427+0000 7f47c9600640 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea91d9d0 0x559cea91fdf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5024aea1eff8b9a2 server_cookie=d1136349db6cf857 in_seq=0 out_seq=0 2024-04-20T21:35:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.428+0000 7f47aaa00640 1 -- 172.21.15.7:0/941332344 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559cea8019e0 con 0x559cea91d9d0 2024-04-20T21:35:44.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.428+0000 7f47aaa00640 10 client.25892 handle_command_reply: tid=0 2024-04-20T21:35:44.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.429+0000 7f47ced68740 1 client.25892 shutdown 2024-04-20T21:35:44.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 msgr2=0x559cea8a1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005d70 tx=0x7f47bc0045f0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 shutdown_connections 2024-04-20T21:35:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cea89d6a0 0x559cea8a1ea0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559cea91d9d0 0x559cea91fdf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cea89b210 0x559cea8a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 --2- 172.21.15.7:0/941332344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559cea8941a0 0x559cea8a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 >> 172.21.15.7:0/941332344 conn(0x559cea88e8d0 msgr2=0x559cea88d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.431+0000 7f47aaa00640 0 client.25892 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:35:44.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.432+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 shutdown_connections 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "03906d40-9b2a-4ca2-8c02-51de7a09f537": { 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "03906d40-9b2a-4ca2-8c02-51de7a09f537", 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:35:44.433 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.432+0000 7f47ced68740 1 -- 172.21.15.7:0/941332344 wait complete. 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.433+0000 7f47ced68740 20 client.25892 trim_cache size 0 max 16384 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 msgr2=0x7f47ac0a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 0x7f47ac0a6170 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47bc04e000 tx=0x7f47bc04b000 comp rx=0 tx=0).stop 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 msgr2=0x7f47c41a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 0x7f47c41a06e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47b0005fd0 tx=0x7f47b000ee50 comp rx=0 tx=0).stop 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 shutdown_connections 2024-04-20T21:35:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47c410c5a0 0x7f47c419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47c4103af0 0x7f47c41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f47ac0a3cb0 0x7f47ac0a6170 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 --2- 172.21.15.7:0/1840474088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47c41031d0 0x7f47c41a06e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 >> 172.21.15.7:0/1840474088 conn(0x7f47c4077f50 msgr2=0x7f47c40fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 shutdown_connections 2024-04-20T21:35:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:35:44.434+0000 7f47cc000640 1 -- 172.21.15.7:0/1840474088 wait complete. 2024-04-20T21:35:44.467 INFO:tasks.cephfs.filesystem:scrub status for tag:03906d40-9b2a-4ca2-8c02-51de7a09f537 - {'path': '/', 'tag': '03906d40-9b2a-4ca2-8c02-51de7a09f537', 'options': 'recursive,force'} 2024-04-20T21:35:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:44 smithi016 ceph-mon[21409]: pgmap v74: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 201 MiB/s wr, 104 op/s 2024-04-20T21:35:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:44 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:44 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:44 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:44.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:44 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:44 smithi186 ceph-mon[22157]: pgmap v74: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 201 MiB/s wr, 104 op/s 2024-04-20T21:35:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:44 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:44 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:44 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:44 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:44 smithi007 ceph-mon[16867]: pgmap v74: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 201 MiB/s wr, 104 op/s 2024-04-20T21:35:44.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:44 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:44.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:44 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:44.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:44 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:44.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:44 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:35:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:35:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:45 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:45 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.786 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:35:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:35:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:45.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:45 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:35:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:46 smithi016 ceph-mon[21409]: pgmap v75: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 192 MiB/s wr, 72 op/s 2024-04-20T21:35:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:46 smithi186 ceph-mon[22157]: pgmap v75: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 192 MiB/s wr, 72 op/s 2024-04-20T21:35:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:46 smithi007 ceph-mon[16867]: pgmap v75: 97 pgs: 97 active+clean; 2.4 GiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 192 MiB/s wr, 72 op/s 2024-04-20T21:35:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:48 smithi016 ceph-mon[21409]: pgmap v76: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 338 MiB/s wr, 119 op/s 2024-04-20T21:35:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:48 smithi186 ceph-mon[22157]: pgmap v76: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 338 MiB/s wr, 119 op/s 2024-04-20T21:35:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:48 smithi007 ceph-mon[16867]: pgmap v76: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 338 MiB/s wr, 119 op/s 2024-04-20T21:35:50.586 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:50 smithi007 ceph-mon[16867]: pgmap v77: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:50 smithi016 ceph-mon[21409]: pgmap v77: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:50 smithi186 ceph-mon[22157]: pgmap v77: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:52 smithi016 ceph-mon[21409]: pgmap v78: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:52 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:52.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:52 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:52 smithi186 ceph-mon[22157]: pgmap v78: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:52 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:52 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:52 smithi007 ceph-mon[16867]: pgmap v78: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 220 MiB/s wr, 79 op/s 2024-04-20T21:35:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:52 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:52 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:53.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:53 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:35:53.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:53 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:35:53.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:53 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:35:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:54 smithi016 ceph-mon[21409]: pgmap v79: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 289 MiB/s wr, 110 op/s 2024-04-20T21:35:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:54 smithi186 ceph-mon[22157]: pgmap v79: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 289 MiB/s wr, 110 op/s 2024-04-20T21:35:54.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:54 smithi007 ceph-mon[16867]: pgmap v79: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 289 MiB/s wr, 110 op/s 2024-04-20T21:35:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:56 smithi016 ceph-mon[21409]: pgmap v80: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 215 MiB/s wr, 78 op/s 2024-04-20T21:35:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:56 smithi186 ceph-mon[22157]: pgmap v80: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 215 MiB/s wr, 78 op/s 2024-04-20T21:35:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:56 smithi007 ceph-mon[16867]: pgmap v80: 97 pgs: 97 active+clean; 4.9 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 215 MiB/s wr, 78 op/s 2024-04-20T21:35:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:35:58 smithi016 ceph-mon[21409]: pgmap v81: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 331 MiB/s wr, 119 op/s 2024-04-20T21:35:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:35:58 smithi186 ceph-mon[22157]: pgmap v81: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 331 MiB/s wr, 119 op/s 2024-04-20T21:35:58.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:35:58 smithi007 ceph-mon[16867]: pgmap v81: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 331 MiB/s wr, 119 op/s 2024-04-20T21:36:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:00 smithi016 ceph-mon[21409]: pgmap v82: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:00 smithi186 ceph-mon[22157]: pgmap v82: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:00 smithi007 ceph-mon[16867]: pgmap v82: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:02 smithi016 ceph-mon[21409]: pgmap v83: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:02 smithi186 ceph-mon[22157]: pgmap v83: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:02 smithi007 ceph-mon[16867]: pgmap v83: 97 pgs: 97 active+clean; 6.2 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 185 MiB/s wr, 72 op/s 2024-04-20T21:36:04.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:04 smithi016 ceph-mon[21409]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 295 MiB/s wr, 127 op/s 2024-04-20T21:36:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:04 smithi186 ceph-mon[22157]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 295 MiB/s wr, 127 op/s 2024-04-20T21:36:04.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:04 smithi007 ceph-mon[16867]: pgmap v84: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 295 MiB/s wr, 127 op/s 2024-04-20T21:36:06.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:06 smithi016 ceph-mon[21409]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 226 MiB/s wr, 96 op/s 2024-04-20T21:36:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:06 smithi186 ceph-mon[22157]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 226 MiB/s wr, 96 op/s 2024-04-20T21:36:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:06 smithi007 ceph-mon[16867]: pgmap v85: 97 pgs: 97 active+clean; 7.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 226 MiB/s wr, 96 op/s 2024-04-20T21:36:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:08 smithi016 ceph-mon[21409]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 325 MiB/s wr, 134 op/s 2024-04-20T21:36:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:08 smithi186 ceph-mon[22157]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 325 MiB/s wr, 134 op/s 2024-04-20T21:36:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:08 smithi007 ceph-mon[16867]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 325 MiB/s wr, 134 op/s 2024-04-20T21:36:10.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:10 smithi016 ceph-mon[21409]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:10 smithi186 ceph-mon[22157]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:10.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:10 smithi007 ceph-mon[16867]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:11.550 INFO:tasks.workunit.client.0.smithi007.stdout:fs setup took 46 secs 2024-04-20T21:36:12.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:12 smithi186 ceph-mon[22157]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:12 smithi007 ceph-mon[16867]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:12.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:12 smithi016 ceph-mon[21409]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 209 MiB/s wr, 93 op/s 2024-04-20T21:36:14.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:14.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:14.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:14 smithi186 ceph-mon[22157]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 342 MiB/s wr, 137 op/s 2024-04-20T21:36:14.745 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:14 smithi016 ceph-mon[21409]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 342 MiB/s wr, 137 op/s 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 -- 172.21.15.7:0/2995341508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc68102f40 msgr2=0x7fcc68103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 --2- 172.21.15.7:0/2995341508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc68102f40 0x7fcc68103320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009900 tx=0x7fcc5802bcc0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 -- 172.21.15.7:0/2995341508 shutdown_connections 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 --2- 172.21.15.7:0/2995341508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc6810c360 0x7fcc6810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 --2- 172.21.15.7:0/2995341508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc6810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 --2- 172.21.15.7:0/2995341508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc68102f40 0x7fcc68103320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 -- 172.21.15.7:0/2995341508 >> 172.21.15.7:0/2995341508 conn(0x7fcc68077a10 msgr2=0x7fcc680fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 -- 172.21.15.7:0/2995341508 shutdown_connections 2024-04-20T21:36:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.755+0000 7fcc67200640 1 -- 172.21.15.7:0/2995341508 wait complete. 2024-04-20T21:36:14.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 Processor -- start 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 -- start start 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68102f40 0x7fcc681a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc681a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68112110 con 0x7fcc68102f40 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68111f90 con 0x7fcc6810c360 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.756+0000 7fcc67200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68112290 con 0x7fcc68103860 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68102f40 0x7fcc681a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46232/0 (socket says 172.21.15.7:46232) 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 -- 172.21.15.7:0/1413091895 learned_addr learned my addr 172.21.15.7:0/1413091895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc681a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 -- 172.21.15.7:0/1413091895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 msgr2=0x7fcc681a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc681a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 -- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68102f40 msgr2=0x7fcc681a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68102f40 0x7fcc681a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 -- 172.21.15.7:0/1413091895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6819afd0 con 0x7fcc6810c360 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.757+0000 7fcc66800640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000eb10 tx=0x7fcc6000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.758+0000 7fcc5ea00640 1 -- 172.21.15.7:0/1413091895 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60004b20 con 0x7fcc6810c360 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.758+0000 7fcc5ea00640 1 -- 172.21.15.7:0/1413091895 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc60004cc0 con 0x7fcc6810c360 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.758+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6819b2c0 con 0x7fcc6810c360 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.758+0000 7fcc5ea00640 1 -- 172.21.15.7:0/1413091895 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60010760 con 0x7fcc6810c360 2024-04-20T21:36:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.758+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc681a7580 con 0x7fcc6810c360 2024-04-20T21:36:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 Processor -- start 2024-04-20T21:36:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc5ea00640 1 -- 172.21.15.7:0/1413091895 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcc60005020 con 0x7fcc6810c360 2024-04-20T21:36:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 -- start start 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f741c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f7425340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f7427c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607f7382150 con 0x5607f741c380 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607f7328370 con 0x5607f7425880 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.761+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5607f737e110 con 0x5607f741cca0 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f741c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f741c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57296/0 (socket says 172.21.15.7:57296) 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc5fe00640 1 -- 172.21.15.7:0/1776809702 learned_addr learned my addr 172.21.15.7:0/1776809702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f7425340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f7427c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 -- 172.21.15.7:0/1776809702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 msgr2=0x5607f7427c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f7427c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 -- 172.21.15.7:0/1776809702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 msgr2=0x5607f741c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f741c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.762+0000 7fcc65e00640 1 -- 172.21.15.7:0/1776809702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f715e7d0 con 0x5607f741cca0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f7425340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc58009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e13c1dd2dea70f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc5ea00640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 0x7fcc4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc57400640 1 -- 172.21.15.7:0/1776809702 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58045070 con 0x5607f741cca0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc5ea00640 1 -- 172.21.15.7:0/1413091895 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc600d0dc0 con 0x7fcc6810c360 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc57400640 1 -- 172.21.15.7:0/1776809702 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5802c660 con 0x5607f741cca0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc57400640 1 -- 172.21.15.7:0/1776809702 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58004b30 con 0x5607f741cca0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.763+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 0x7fcc4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1776809702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 msgr2=0x5607f7425340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f7425340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc58009670 comp rx=0 tx=0).stop 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1776809702 shutdown_connections 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f7427c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 0x7fcc4c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5607f7417c40 tx=0x7fcc50015040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f7425340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1776809702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f741c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1776809702 >> 172.21.15.7:0/1776809702 conn(0x5607f7416ab0 msgr2=0x5607f7416e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1776809702 shutdown_connections 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1776809702 wait complete. 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.764+0000 7fcc6fe21740 1 -- start start 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 Processor -- start 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f74228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f7423330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5607f7428610 con 0x5607f741c380 2024-04-20T21:36:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5607f733b740 con 0x5607f741cca0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5607f7428790 con 0x5607f7425880 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc6fe21740 10 client.0 _ll_register_callbacks cb 0x5607f7336b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.765+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46250/0 (socket says 172.21.15.7:46250) 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc66800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f7423330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 -- 172.21.15.7:0/1772402100 learned_addr learned my addr 172.21.15.7:0/1772402100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f74228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 -- 172.21.15.7:0/1772402100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 msgr2=0x5607f7423330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f7423330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 -- 172.21.15.7:0/1772402100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 msgr2=0x5607f74228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f74228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 -- 172.21.15.7:0/1772402100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f70943f0 con 0x5607f741cca0 2024-04-20T21:36:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc66800640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f7423330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc50011a40 tx=0x7fcc50005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867733735dcb0db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.766+0000 7fcc4be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500229b0 con 0x5607f741cca0 2024-04-20T21:36:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc50022b50 con 0x5607f741cca0 2024-04-20T21:36:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5802c660 con 0x5607f741cca0 2024-04-20T21:36:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc50022d50 con 0x5607f741cca0 2024-04-20T21:36:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc6fe21740 10 client.25904 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607f70540f0 con 0x5607f741cca0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.767+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5802c660 con 0x5607f741cca0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fcc5002c480 con 0x5607f741cca0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 10 client.25904 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 20 client.25904 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f7435ca0 0x5607f74380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 4 client.25904 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f738fb70 con 0x5607f7435ca0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.768+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f7435ca0 0x5607f74380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.770+0000 7fcc65e00640 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f7435ca0 0x5607f74380f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471d19f572b37102 server_cookie=3b9bd07e76bbc082 in_seq=0 out_seq=0 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.770+0000 7fcc4be00640 10 client.25904 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc4be00640 1 -- 172.21.15.7:0/1772402100 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607f738fb70 con 0x5607f7435ca0 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc4be00640 10 client.25904 handle_command_reply: tid=0 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 client.25904 shutdown 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 msgr2=0x5607f7422df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcc50011a40 tx=0x7fcc50005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 shutdown_connections 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f7422df0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f7435ca0 0x5607f74380f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f7423330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/1772402100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f74228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc4be00640 0 client.25904 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 >> 172.21.15.7:0/1772402100 conn(0x5607f7416ab0 msgr2=0x5607f7418800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 shutdown_connections 2024-04-20T21:36:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 1 -- 172.21.15.7:0/1772402100 wait complete. 2024-04-20T21:36:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.771+0000 7fcc6fe21740 20 client.25904 trim_cache size 0 max 16384 2024-04-20T21:36:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:14 smithi007 ceph-mon[16867]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 342 MiB/s wr, 137 op/s 2024-04-20T21:36:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 Processor -- start 2024-04-20T21:36:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 -- start start 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f743da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681b2490 con 0x5607f741c380 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681afd50 con 0x5607f741cca0 2024-04-20T21:36:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681adba0 con 0x5607f7425880 2024-04-20T21:36:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46258/0 (socket says 172.21.15.7:46258) 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc5fe00640 1 -- 172.21.15.7:0/635701491 learned_addr learned my addr 172.21.15.7:0/635701491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.791+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57322/0 (socket says 172.21.15.7:57322) 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc66800640 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f743da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc5fe00640 1 -- 172.21.15.7:0/635701491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 msgr2=0x5607f743da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f743da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc5fe00640 1 -- 172.21.15.7:0/635701491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 msgr2=0x5607f7424fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc5fe00640 1 -- 172.21.15.7:0/635701491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f741cad0 con 0x5607f741cca0 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.792+0000 7fcc65e00640 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.793+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009c80 tx=0x7fcc50011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972b643ce6d8973d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.793+0000 7fcc56a00640 1 -- 172.21.15.7:0/635701491 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50039070 con 0x5607f741cca0 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.793+0000 7fcc56a00640 1 -- 172.21.15.7:0/635701491 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5002cd30 con 0x5607f741cca0 2024-04-20T21:36:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.793+0000 7fcc56a00640 1 -- 172.21.15.7:0/635701491 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500359a0 con 0x5607f741cca0 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 -- 172.21.15.7:0/635701491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 msgr2=0x5607f74314d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009c80 tx=0x7fcc50011e40 comp rx=0 tx=0).stop 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 -- 172.21.15.7:0/635701491 shutdown_connections 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f741cca0 0x5607f74314d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f7425880 0x5607f743da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/635701491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f7424fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 -- 172.21.15.7:0/635701491 >> 172.21.15.7:0/635701491 conn(0x5607f7416ab0 msgr2=0x5607f7416e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 -- 172.21.15.7:0/635701491 shutdown_connections 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.794+0000 7fcc6fe21740 1 -- 172.21.15.7:0/635701491 wait complete. 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 -- start start 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 Processor -- start 2024-04-20T21:36:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f742de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f741f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681ae210 con 0x5607f741c380 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681af680 con 0x5607f7425880 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681ae8e0 con 0x5607f741cca0 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.795+0000 7fcc6fe21740 10 client.0 _ll_register_callbacks cb 0x7fcc50009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f742de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46266/0 (socket says 172.21.15.7:46266) 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc66800640 1 -- 172.21.15.7:0/2895760378 learned_addr learned my addr 172.21.15.7:0/2895760378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f741f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 -- 172.21.15.7:0/2895760378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 msgr2=0x5607f741f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f741f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 -- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 msgr2=0x5607f741f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc65e00640 1 -- 172.21.15.7:0/2895760378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f7420110 con 0x5607f741c380 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.796+0000 7fcc66800640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f742de70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc5802c200 tx=0x7fcc58005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9f6534d4e021a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc4be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58045040 con 0x5607f741c380 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc580048c0 con 0x5607f741c380 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5002cd30 con 0x5607f741c380 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc5802c720 con 0x5607f741c380 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc6fe21740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.797+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607f749c320 con 0x5607f741c380 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5002cd30 con 0x5607f741c380 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fcc5804a450 con 0x5607f741c380 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 20 client.15780 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f749c720 0x5607f749eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 4 client.15780 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.798+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607f73899e0 con 0x5607f749c720 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.799+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f749c720 0x5607f749eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.799+0000 7fcc4be00640 10 client.15780 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.799+0000 7fcc5fe00640 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f749c720 0x5607f749eb10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e1d5b663a79622 server_cookie=8b6a08f10f3c4336 in_seq=0 out_seq=0 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc4be00640 1 -- 172.21.15.7:0/2895760378 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607f73899e0 con 0x5607f749c720 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc4be00640 10 client.15780 handle_command_reply: tid=0 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 client.15780 shutdown 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 msgr2=0x5607f742de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f742de70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc5802c200 tx=0x7fcc58005fd0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 shutdown_connections 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5607f7425880 0x5607f741f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5607f749c720 0x5607f749eb10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607f741cca0 0x5607f741f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 --2- 172.21.15.7:0/2895760378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5607f741c380 0x5607f742de70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 >> 172.21.15.7:0/2895760378 conn(0x5607f7416ab0 msgr2=0x5607f7422870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.800+0000 7fcc4be00640 0 client.15780 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.804+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 shutdown_connections 2024-04-20T21:36:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.805+0000 7fcc6fe21740 1 -- 172.21.15.7:0/2895760378 wait complete. 2024-04-20T21:36:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.805+0000 7fcc6fe21740 20 client.15780 trim_cache size 0 max 16384 2024-04-20T21:36:14.804 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:36:14.804 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:36:14.804 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 msgr2=0x7fcc4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 0x7fcc4c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5607f7417c40 tx=0x7fcc50015040 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 msgr2=0x7fcc6819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000eb10 tx=0x7fcc6000c2d0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 shutdown_connections 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc4c0a3d20 0x7fcc4c0a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc6810c360 0x7fcc6819a710 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc68103860 0x7fcc681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 --2- 172.21.15.7:0/1413091895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68102f40 0x7fcc681a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.806+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 >> 172.21.15.7:0/1413091895 conn(0x7fcc68077a10 msgr2=0x7fcc68100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.807+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 shutdown_connections 2024-04-20T21:36:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:14.807+0000 7fcc67200640 1 -- 172.21.15.7:0/1413091895 wait complete. 2024-04-20T21:36:14.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:36:14.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:15.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 -- 172.21.15.7:0/4140553038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0109490 msgr2=0x7f80e0111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 --2- 172.21.15.7:0/4140553038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0109490 0x7f80e0111060 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009be0 tx=0x7f80d802bef0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 -- 172.21.15.7:0/4140553038 shutdown_connections 2024-04-20T21:36:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 --2- 172.21.15.7:0/4140553038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e0109490 0x7f80e0111060 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 --2- 172.21.15.7:0/4140553038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80e0104d70 0x7f80e0108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 --2- 172.21.15.7:0/4140553038 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e01043c0 0x7f80e01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.150+0000 7f80e8200640 1 -- 172.21.15.7:0/4140553038 >> 172.21.15.7:0/4140553038 conn(0x7f80e00ffc20 msgr2=0x7f80e0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.151+0000 7f80e8200640 1 -- 172.21.15.7:0/4140553038 shutdown_connections 2024-04-20T21:36:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.151+0000 7f80e8200640 1 -- 172.21.15.7:0/4140553038 wait complete. 2024-04-20T21:36:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.152+0000 7f80e8200640 1 Processor -- start 2024-04-20T21:36:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.152+0000 7f80e8200640 1 -- start start 2024-04-20T21:36:15.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e0109490 0x7f80e01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80e019a860 0x7f80e019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80e0114520 con 0x7f80e019a860 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80e01143a0 con 0x7f80e01043c0 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e8200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f80e01146a0 con 0x7f80e0109490 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46276/0 (socket says 172.21.15.7:46276) 2024-04-20T21:36:15.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 -- 172.21.15.7:0/20473746 learned_addr learned my addr 172.21.15.7:0/20473746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 -- 172.21.15.7:0/20473746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e0109490 msgr2=0x7f80e01a0be0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:36:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e0109490 0x7f80e01a0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 -- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80e019a860 msgr2=0x7f80e019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80e019a860 0x7f80e019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.153+0000 7f80e5800640 1 -- 172.21.15.7:0/20473746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e019b3f0 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80e5800640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80d400e9f0 tx=0x7f80d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d400cd80 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d4010910 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4018980 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e019b6e0 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.154+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e01a77b0 con 0x7f80e01043c0 2024-04-20T21:36:15.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.156+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e0105890 con 0x7f80e01043c0 2024-04-20T21:36:15.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.158+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f80d4010ac0 con 0x7f80e01043c0 2024-04-20T21:36:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.159+0000 7f80de000640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 0x7f80c80a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.160+0000 7f80e4e00640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 0x7f80c80a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.160+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f80d40d0c00 con 0x7f80e01043c0 2024-04-20T21:36:15.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.161+0000 7f80e4e00640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 0x7f80c80a6220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80e01033e0 tx=0x7f80cc009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.167+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f80d40d7080 con 0x7f80e01043c0 2024-04-20T21:36:15.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.564+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80e0104d70 con 0x7f80e01043c0 2024-04-20T21:36:15.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.567+0000 7f80de000640 1 -- 172.21.15.7:0/20473746 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7f80e019b6e0 con 0x7f80e01043c0 2024-04-20T21:36:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:36:15.568 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:35:06.444983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24559,24575,24535]},"id":1}]} 2024-04-20T21:36:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-04-20T21:36:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 msgr2=0x7f80c80a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 0x7f80c80a6220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80e01033e0 tx=0x7f80cc009260 comp rx=0 tx=0).stop 2024-04-20T21:36:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 msgr2=0x7f80e01a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f80d400e9f0 tx=0x7f80d400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 shutdown_connections 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80e019a860 0x7f80e019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80e0109490 0x7f80e01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80c80a3d60 0x7f80c80a6220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 --2- 172.21.15.7:0/20473746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80e01043c0 0x7f80e01a06a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.572+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 >> 172.21.15.7:0/20473746 conn(0x7f80e00ffc20 msgr2=0x7f80e0101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.573+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 shutdown_connections 2024-04-20T21:36:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.573+0000 7f80e8200640 1 -- 172.21.15.7:0/20473746 wait complete. 2024-04-20T21:36:15.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 -- 172.21.15.7:0/671239485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c10e7e0 msgr2=0x7fe28c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 --2- 172.21.15.7:0/671239485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c10e7e0 0x7fe28c110bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe284009ca0 tx=0x7fe28402c190 comp rx=0 tx=0).stop 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 -- 172.21.15.7:0/671239485 shutdown_connections 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 --2- 172.21.15.7:0/671239485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c10e7e0 0x7fe28c110bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 --2- 172.21.15.7:0/671239485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c074c30 0x7fe28c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 --2- 172.21.15.7:0/671239485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074310 0x7fe28c0746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 -- 172.21.15.7:0/671239485 >> 172.21.15.7:0/671239485 conn(0x7fe28c0fda10 msgr2=0x7fe28c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 -- 172.21.15.7:0/671239485 shutdown_connections 2024-04-20T21:36:15.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.919+0000 7fe292600640 1 -- 172.21.15.7:0/671239485 wait complete. 2024-04-20T21:36:15.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.921+0000 7fe292600640 1 Processor -- start 2024-04-20T21:36:15.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.921+0000 7fe292600640 1 -- start start 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.921+0000 7fe292600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c074310 0x7fe28c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.921+0000 7fe292600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 0x7fe28c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe292600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe292600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1144c0 con 0x7fe28c074c30 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe292600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c114340 con 0x7fe28c074310 2024-04-20T21:36:15.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe292600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c114640 con 0x7fe28c10e7e0 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34258/0 (socket says 172.21.15.7:34258) 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 -- 172.21.15.7:0/1765752386 learned_addr learned my addr 172.21.15.7:0/1765752386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28b400640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c074310 0x7fe28c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 -- 172.21.15.7:0/1765752386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c074310 msgr2=0x7fe28c1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c074310 0x7fe28c1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28be00640 1 -- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 msgr2=0x7fe28c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.922+0000 7fe28aa00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 0x7fe28c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.923+0000 7fe28be00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 0x7fe28c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.923+0000 7fe28be00640 1 -- 172.21.15.7:0/1765752386 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c19b010 con 0x7fe28c10e7e0 2024-04-20T21:36:15.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.923+0000 7fe28aa00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 0x7fe28c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.923+0000 7fe28be00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe284037040 tx=0x7fe284035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.923+0000 7fe283e00640 1 -- 172.21.15.7:0/1765752386 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284039040 con 0x7fe28c10e7e0 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.924+0000 7fe283e00640 1 -- 172.21.15.7:0/1765752386 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe284044070 con 0x7fe28c10e7e0 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.924+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c19b2a0 con 0x7fe28c10e7e0 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.924+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c1a74d0 con 0x7fe28c10e7e0 2024-04-20T21:36:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.924+0000 7fe283e00640 1 -- 172.21.15.7:0/1765752386 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284004860 con 0x7fe28c10e7e0 2024-04-20T21:36:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.930+0000 7fe295386740 1 Processor -- start 2024-04-20T21:36:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 -- start start 2024-04-20T21:36:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe283e00640 1 -- 172.21.15.7:0/1765752386 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe284003b70 con 0x7fe28c10e7e0 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x5645405f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x5645406070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564540558050 con 0x5645405f3340 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5645404fe490 con 0x564540604cc0 2024-04-20T21:36:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.936+0000 7fe295386740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564540554010 con 0x5645405f2990 2024-04-20T21:36:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.938+0000 7fe28b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57364/0 (socket says 172.21.15.7:57364) 2024-04-20T21:36:15.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 -- 172.21.15.7:0/706436233 learned_addr learned my addr 172.21.15.7:0/706436233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28aa00640 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x5645406070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 -- 172.21.15.7:0/706436233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 msgr2=0x5645405f2d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x5645405f2d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 -- 172.21.15.7:0/706436233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 msgr2=0x5645406070b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x5645406070b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28b400640 1 -- 172.21.15.7:0/706436233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564540506070 con 0x5645405f3340 2024-04-20T21:36:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.941+0000 7fe28aa00640 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x5645406070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.942+0000 7fe283e00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 0x7fe2700a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.942+0000 7fe283e00640 1 -- 172.21.15.7:0/1765752386 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe2840f2b30 con 0x7fe28c10e7e0 2024-04-20T21:36:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.942+0000 7fe28b400640 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009c80 tx=0x7fe27c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ed32348f4f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.943+0000 7fe280c00640 1 -- 172.21.15.7:0/706436233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c012450 con 0x5645405f3340 2024-04-20T21:36:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.943+0000 7fe280c00640 1 -- 172.21.15.7:0/706436233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe27c012ad0 con 0x5645405f3340 2024-04-20T21:36:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.943+0000 7fe280c00640 1 -- 172.21.15.7:0/706436233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00fc70 con 0x5645405f3340 2024-04-20T21:36:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 -- 172.21.15.7:0/706436233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 msgr2=0x564540604780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009c80 tx=0x7fe27c0117b0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 -- 172.21.15.7:0/706436233 shutdown_connections 2024-04-20T21:36:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x5645406070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f3340 0x564540604780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 --2- 172.21.15.7:0/706436233 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x5645405f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.944+0000 7fe295386740 1 -- 172.21.15.7:0/706436233 >> 172.21.15.7:0/706436233 conn(0x5645405ed6e0 msgr2=0x5645405edac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.945+0000 7fe295386740 1 -- 172.21.15.7:0/706436233 shutdown_connections 2024-04-20T21:36:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.945+0000 7fe295386740 1 -- 172.21.15.7:0/706436233 wait complete. 2024-04-20T21:36:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.945+0000 7fe295386740 1 -- start start 2024-04-20T21:36:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.945+0000 7fe28aa00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 0x7fe2700a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 Processor -- start 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645405f3340 0x564540603a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564540604cc0 0x564540603fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564540607a80 con 0x564540604cc0 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564540511860 con 0x5645405f3340 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564540607c00 con 0x5645405f2990 2024-04-20T21:36:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.946+0000 7fe295386740 10 client.0 _ll_register_callbacks cb 0x5645404dffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.947+0000 7fe28b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.947+0000 7fe28b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34286/0 (socket says 172.21.15.7:34286) 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.947+0000 7fe28b400640 1 -- 172.21.15.7:0/288948272 learned_addr learned my addr 172.21.15.7:0/288948272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.947+0000 7fe28be00640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564540604cc0 0x564540603fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.947+0000 7fe28aa00640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645405f3340 0x564540603a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28aa00640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 0x7fe2700a62e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5645405eeb00 tx=0x7fe278005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 -- 172.21.15.7:0/288948272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645405f3340 msgr2=0x564540603a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645405f3340 0x564540603a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 -- 172.21.15.7:0/288948272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564540604cc0 msgr2=0x564540603fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564540604cc0 0x564540603fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 -- 172.21.15.7:0/288948272 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564540515950 con 0x5645405f2990 2024-04-20T21:36:15.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe28b400640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00f480 tx=0x7fe27c009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3ed6c69c124a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.948+0000 7fe275600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:15.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.949+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c012450 con 0x5645405f2990 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.949+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27c012ad0 con 0x5645405f2990 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.949+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe27c0125f0 con 0x5645405f2990 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.950+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe27c032050 con 0x5645405f2990 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.950+0000 7fe295386740 10 client.25900 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.950+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5645404e3820 con 0x5645405f2990 2024-04-20T21:36:15.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.950+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe27c012ad0 con 0x5645405f2990 2024-04-20T21:36:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fe27c005450 con 0x5645405f2990 2024-04-20T21:36:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 10 client.25900 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 10 client.25900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:36:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 20 client.25900 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:15.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564540614bf0 0x564540616fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 4 client.25900 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:15.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564540565a70 con 0x564540614bf0 2024-04-20T21:36:15.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.952+0000 7fe28aa00640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564540614bf0 0x564540616fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.953+0000 7fe28aa00640 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564540614bf0 0x564540616fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc834215e8c5cf0a server_cookie=d7819257995a8156 in_seq=0 out_seq=0 2024-04-20T21:36:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.953+0000 7fe275600640 10 client.25900 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:15.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.954+0000 7fe275600640 1 -- 172.21.15.7:0/288948272 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564540565a70 con 0x564540614bf0 2024-04-20T21:36:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.954+0000 7fe275600640 10 client.25900 handle_command_reply: tid=0 2024-04-20T21:36:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.954+0000 7fe295386740 1 client.25900 shutdown 2024-04-20T21:36:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.954+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 msgr2=0x564540603530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.954+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00f480 tx=0x7fe27c009fa0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 shutdown_connections 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564540614bf0 0x564540616fe0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5645405f3340 0x564540603a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564540604cc0 0x564540603fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 --2- 172.21.15.7:0/288948272 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f2990 0x564540603530 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 >> 172.21.15.7:0/288948272 conn(0x5645405ed6e0 msgr2=0x5645405ef430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe275600640 0 client.25900 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 shutdown_connections 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 1 -- 172.21.15.7:0/288948272 wait complete. 2024-04-20T21:36:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.955+0000 7fe295386740 20 client.25900 trim_cache size 0 max 16384 2024-04-20T21:36:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.975+0000 7fe295386740 1 Processor -- start 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.975+0000 7fe295386740 1 -- start start 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x5645405ef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x564540604240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1ae280 con 0x5645405f2990 2024-04-20T21:36:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1abb40 con 0x564540604cc0 2024-04-20T21:36:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.976+0000 7fe295386740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1a9990 con 0x5645405f3340 2024-04-20T21:36:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.977+0000 7fe28be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.977+0000 7fe28aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x564540604240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.977+0000 7fe28be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46310/0 (socket says 172.21.15.7:46310) 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.977+0000 7fe28be00640 1 -- 172.21.15.7:0/297807457 learned_addr learned my addr 172.21.15.7:0/297807457 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.977+0000 7fe28aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x564540604240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34300/0 (socket says 172.21.15.7:34300) 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.978+0000 7fe28b400640 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x5645405ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28be00640 1 -- 172.21.15.7:0/297807457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 msgr2=0x564540604240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28be00640 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x564540604240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28be00640 1 -- 172.21.15.7:0/297807457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 msgr2=0x5645405ef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28be00640 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x5645405ef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28be00640 1 -- 172.21.15.7:0/297807457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564540607420 con 0x564540604cc0 2024-04-20T21:36:15.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.979+0000 7fe28b400640 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x5645405ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:15.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.980+0000 7fe28be00640 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2840be8d0 tx=0x7fe2840be7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b510154b940ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.984+0000 7fe277e00640 1 -- 172.21.15.7:0/297807457 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840bebd0 con 0x564540604cc0 2024-04-20T21:36:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.984+0000 7fe277e00640 1 -- 172.21.15.7:0/297807457 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2840c2f80 con 0x564540604cc0 2024-04-20T21:36:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.984+0000 7fe277e00640 1 -- 172.21.15.7:0/297807457 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840333a0 con 0x564540604cc0 2024-04-20T21:36:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.985+0000 7fe295386740 1 -- 172.21.15.7:0/297807457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 msgr2=0x564540602640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.985+0000 7fe295386740 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2840be8d0 tx=0x7fe2840be7d0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 -- 172.21.15.7:0/297807457 shutdown_connections 2024-04-20T21:36:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540602640 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x564540604240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 --2- 172.21.15.7:0/297807457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x5645405ef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 -- 172.21.15.7:0/297807457 >> 172.21.15.7:0/297807457 conn(0x5645405ed6e0 msgr2=0x564540600dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.987+0000 7fe295386740 1 -- 172.21.15.7:0/297807457 shutdown_connections 2024-04-20T21:36:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.988+0000 7fe295386740 1 -- 172.21.15.7:0/297807457 wait complete. 2024-04-20T21:36:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.989+0000 7fe295386740 1 -- start start 2024-04-20T21:36:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.989+0000 7fe295386740 1 Processor -- start 2024-04-20T21:36:15.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.990+0000 7fe295386740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x56454060d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.990+0000 7fe295386740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x5645406085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe295386740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540608ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe28be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540608ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe28be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540608ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46322/0 (socket says 172.21.15.7:46322) 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe28be00640 1 -- 172.21.15.7:0/3694835594 learned_addr learned my addr 172.21.15.7:0/3694835594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe28aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x5645406085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1aa000 con 0x5645405f2990 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1ab470 con 0x564540604cc0 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.991+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1aa6d0 con 0x5645405f3340 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 -- 172.21.15.7:0/3694835594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 msgr2=0x564540608ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540608ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 -- 172.21.15.7:0/3694835594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 msgr2=0x56454060d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x56454060d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454026a3f0 con 0x5645405f3340 2024-04-20T21:36:15.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe295386740 10 client.0 _ll_register_callbacks cb 0x7fe27c004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:15.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2840c2f80 con 0x5645405f3340 2024-04-20T21:36:15.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe28aa00640 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x5645406085a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x564540601600 tx=0x7fe2780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dc12899b19e53c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.992+0000 7fe275600640 10 client.25906 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.993+0000 7fe275600640 1 -- 172.21.15.7:0/3694835594 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278027550 con 0x5645405f3340 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.993+0000 7fe275600640 1 -- 172.21.15.7:0/3694835594 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe2780276f0 con 0x5645405f3340 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.993+0000 7fe275600640 1 -- 172.21.15.7:0/3694835594 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe2780278f0 con 0x5645405f3340 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.994+0000 7fe295386740 10 client.25906 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.994+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5645403347d0 con 0x5645405f3340 2024-04-20T21:36:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.994+0000 7fe275600640 1 -- 172.21.15.7:0/3694835594 <== mon.1 v2:172.21.15.186:3300/0 4 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fe27801e430 con 0x5645405f3340 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 10 client.25906 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 10 client.25906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 20 client.25906 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56454067afc0 0x56454067d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 4 client.25906 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.995+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454055f8e0 con 0x56454067afc0 2024-04-20T21:36:15.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.996+0000 7fe28b400640 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56454067afc0 0x56454067d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:15.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.996+0000 7fe28b400640 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56454067afc0 0x56454067d410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3147b184ee91154b server_cookie=17f17cca7eaa0c3f in_seq=0 out_seq=0 2024-04-20T21:36:15.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.996+0000 7fe275600640 10 client.25906 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:15.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.996+0000 7fe275600640 1 -- 172.21.15.7:0/3694835594 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454055f8e0 con 0x56454067afc0 2024-04-20T21:36:15.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.996+0000 7fe275600640 10 client.25906 handle_command_reply: tid=0 2024-04-20T21:36:15.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.997+0000 7fe295386740 1 client.25906 shutdown 2024-04-20T21:36:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 msgr2=0x5645406085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x5645406085a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x564540601600 tx=0x7fe2780046c0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 shutdown_connections 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56454067afc0 0x56454067d410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564540604cc0 0x564540608ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5645405f3340 0x5645406085a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 --2- 172.21.15.7:0/3694835594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5645405f2990 0x56454060d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 >> 172.21.15.7:0/3694835594 conn(0x5645405ed6e0 msgr2=0x5645405ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe275600640 0 client.25906 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 shutdown_connections 2024-04-20T21:36:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 1 -- 172.21.15.7:0/3694835594 wait complete. 2024-04-20T21:36:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.998+0000 7fe295386740 20 client.25906 trim_cache size 0 max 16384 2024-04-20T21:36:15.998 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:15.999+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 msgr2=0x7fe2700a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.000+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 0x7fe2700a62e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5645405eeb00 tx=0x7fe278005f90 comp rx=0 tx=0).stop 2024-04-20T21:36:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.000+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 msgr2=0x7fe28c19a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.000+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe284037040 tx=0x7fe284035d10 comp rx=0 tx=0).stop 2024-04-20T21:36:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 shutdown_connections 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe28c10e7e0 0x7fe28c19a840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28c074c30 0x7fe28c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe2700a3e20 0x7fe2700a62e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 --2- 172.21.15.7:0/1765752386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28c074310 0x7fe28c1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 >> 172.21.15.7:0/1765752386 conn(0x7fe28c0fda10 msgr2=0x7fe28c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 shutdown_connections 2024-04-20T21:36:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.001+0000 7fe292600640 1 -- 172.21.15.7:0/1765752386 wait complete. 2024-04-20T21:36:16.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.296+0000 7f832a000640 1 -- 172.21.15.7:0/1590125074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8324100fc0 msgr2=0x7f83241013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.296+0000 7f832a000640 1 --2- 172.21.15.7:0/1590125074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8324100fc0 0x7f83241013a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402bcd0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 -- 172.21.15.7:0/1590125074 shutdown_connections 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 --2- 172.21.15.7:0/1590125074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83241018e0 0x7f8324105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 --2- 172.21.15.7:0/1590125074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241062e0 0x7f8324106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 --2- 172.21.15.7:0/1590125074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8324100fc0 0x7f83241013a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 -- 172.21.15.7:0/1590125074 >> 172.21.15.7:0/1590125074 conn(0x7f8324078010 msgr2=0x7f8324078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 -- 172.21.15.7:0/1590125074 shutdown_connections 2024-04-20T21:36:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.297+0000 7f832a000640 1 -- 172.21.15.7:0/1590125074 wait complete. 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.298+0000 7f832a000640 1 Processor -- start 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.298+0000 7f832a000640 1 -- start start 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 0x7f832419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83241062e0 0x7f83241a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83241189a0 con 0x7f83241062e0 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8324118820 con 0x7f8324100fc0 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f832a000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8324118b20 con 0x7f83241018e0 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f8322a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f8322a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34324/0 (socket says 172.21.15.7:34324) 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f8322a00640 1 -- 172.21.15.7:0/2297212992 learned_addr learned my addr 172.21.15.7:0/2297212992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.299+0000 7f8323400640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 0x7f832419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323e00640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83241062e0 0x7f83241a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 -- 172.21.15.7:0/2297212992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 msgr2=0x7f832419f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 -- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83241062e0 msgr2=0x7f83241a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83241062e0 0x7f83241a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 -- 172.21.15.7:0/2297212992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a3df0 con 0x7f8324100fc0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8322a00640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.300+0000 7f8323400640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 0x7f832419ee00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f831402c210 tx=0x7f831402c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.301+0000 7f831be00640 1 -- 172.21.15.7:0/2297212992 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314043020 con 0x7f8324100fc0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.301+0000 7f831be00640 1 -- 172.21.15.7:0/2297212992 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83140054e0 con 0x7f8324100fc0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.301+0000 7f831be00640 1 -- 172.21.15.7:0/2297212992 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314003d40 con 0x7f8324100fc0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.302+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a4080 con 0x7f8324100fc0 2024-04-20T21:36:16.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.302+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241ab970 con 0x7f8324100fc0 2024-04-20T21:36:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.307+0000 7f832cdd8740 1 Processor -- start 2024-04-20T21:36:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.307+0000 7f831be00640 1 -- 172.21.15.7:0/2297212992 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8314046020 con 0x7f8324100fc0 2024-04-20T21:36:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.307+0000 7f832cdd8740 1 -- start start 2024-04-20T21:36:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc01390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc0c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcb66050 con 0x55dfdcc00fb0 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcb0c490 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f832cdd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcb62010 con 0x55dfdcc018d0 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.308+0000 7f8323e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc01390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34340/0 (socket says 172.21.15.7:34340) 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8323400640 1 -- 172.21.15.7:0/3177554962 learned_addr learned my addr 172.21.15.7:0/3177554962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc0c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f831be00640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 0x7f83080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 -- 172.21.15.7:0/3177554962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 msgr2=0x55dfdcc09f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 -- 172.21.15.7:0/3177554962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 msgr2=0x55dfdcc01390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc01390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f8322a00640 1 -- 172.21.15.7:0/3177554962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdcb14070 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.309+0000 7f831be00640 1 -- 172.21.15.7:0/2297212992 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f83140f2230 con 0x7f8324100fc0 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.310+0000 7f8323400640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.310+0000 7f8323e00640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc01390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.310+0000 7f8322a00640 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc0c8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8310009df0 tx=0x7f831000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee068a3e26be562 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.310+0000 7f8323e00640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 0x7f83080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.311+0000 7f830fe00640 1 -- 172.21.15.7:0/3177554962 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310017070 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.311+0000 7f830fe00640 1 -- 172.21.15.7:0/3177554962 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310013950 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.311+0000 7f830fe00640 1 -- 172.21.15.7:0/3177554962 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310012900 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.311+0000 7f8323e00640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 0x7f83080a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f831c0125f0 tx=0x7f831c004a40 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.312+0000 7f832cdd8740 1 -- 172.21.15.7:0/3177554962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 msgr2=0x55dfdcc0c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.312+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc0c8a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8310009df0 tx=0x7f831000e5a0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.313+0000 7f832cdd8740 1 -- 172.21.15.7:0/3177554962 shutdown_connections 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.313+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc0c8a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.313+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc018d0 0x55dfdcc09f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.313+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3177554962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc01390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.313+0000 7f832cdd8740 1 -- 172.21.15.7:0/3177554962 >> 172.21.15.7:0/3177554962 conn(0x55dfdcbfb6e0 msgr2=0x55dfdcbfbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.314+0000 7f832cdd8740 1 -- 172.21.15.7:0/3177554962 shutdown_connections 2024-04-20T21:36:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.314+0000 7f832cdd8740 1 -- 172.21.15.7:0/3177554962 wait complete. 2024-04-20T21:36:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.314+0000 7f832cdd8740 1 -- start start 2024-04-20T21:36:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 Processor -- start 2024-04-20T21:36:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcc0d2a0 con 0x55dfdcc018d0 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcb1f860 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55dfdcc0d420 con 0x55dfdcc00fb0 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.315+0000 7f832cdd8740 10 client.0 _ll_register_callbacks cb 0x55dfdcaedfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.316+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.316+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34342/0 (socket says 172.21.15.7:34342) 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.316+0000 7f8323400640 1 -- 172.21.15.7:0/1117457583 learned_addr learned my addr 172.21.15.7:0/1117457583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.317+0000 7f8322a00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.317+0000 7f8323400640 1 -- 172.21.15.7:0/1117457583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 msgr2=0x55dfdcc07f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:36:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8323e00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8323400640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8323400640 1 -- 172.21.15.7:0/1117457583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 msgr2=0x55dfdcc079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8323400640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc079e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8323400640 1 -- 172.21.15.7:0/1117457583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdcb23950 con 0x55dfdcc00fb0 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.319+0000 7f8322a00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f8323e00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f8323400640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83140baca0 tx=0x7f8314005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ab31957ddbebf12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f830cc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140bde10 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f83140efbf0 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.320+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8310013950 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.321+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83140bdc30 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.323+0000 7f832cdd8740 10 client.25915 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.323+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfdcaf1820 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310013950 con 0x55dfdcc00fb0 2024-04-20T21:36:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f83140c6ab0 con 0x55dfdcc00fb0 2024-04-20T21:36:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 10 client.25915 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 10 client.25915 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:36:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 20 client.25915 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc1a3d0 0x55dfdcc1c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 4 client.25915 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfdcb73a70 con 0x55dfdcc1a3d0 2024-04-20T21:36:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.324+0000 7f8322a00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc1a3d0 0x55dfdcc1c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.326+0000 7f8322a00640 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc1a3d0 0x55dfdcc1c820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32eec89e69afd0dc server_cookie=59b0984bcafc4c91 in_seq=0 out_seq=0 2024-04-20T21:36:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.326+0000 7f830cc00640 10 client.25915 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.327+0000 7f830cc00640 1 -- 172.21.15.7:0/1117457583 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfdcb73a70 con 0x55dfdcc1a3d0 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.327+0000 7f830cc00640 10 client.25915 handle_command_reply: tid=0 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.327+0000 7f832cdd8740 1 client.25915 shutdown 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 msgr2=0x55dfdcc074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83140baca0 tx=0x7f8314005bd0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 shutdown_connections 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc1a3d0 0x55dfdcc1c820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1117457583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc00fb0 0x55dfdcc074a0 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83140baca0 tx=0x7f8314005bd0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 >> 172.21.15.7:0/1117457583 conn(0x55dfdcbfb6e0 msgr2=0x55dfdcbfd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f830cc00640 0 client.25915 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 shutdown_connections 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 1 -- 172.21.15.7:0/1117457583 wait complete. 2024-04-20T21:36:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.328+0000 7f832cdd8740 20 client.25915 trim_cache size 0 max 16384 2024-04-20T21:36:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.353+0000 7f832cdd8740 1 Processor -- start 2024-04-20T21:36:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.353+0000 7f832cdd8740 1 -- start start 2024-04-20T21:36:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b69b0 con 0x55dfdcc018d0 2024-04-20T21:36:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b4270 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f832cdd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b20c0 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f8323400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f8323400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57440/0 (socket says 172.21.15.7:57440) 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f8323400640 1 -- 172.21.15.7:0/3667726520 learned_addr learned my addr 172.21.15.7:0/3667726520 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.354+0000 7f8322a00640 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323e00640 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 -- 172.21.15.7:0/3667726520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc0a4b0 msgr2=0x55dfdcc07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 -- 172.21.15.7:0/3667726520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcbfa5c0 msgr2=0x55dfdcc220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 -- 172.21.15.7:0/3667726520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfdc8783f0 con 0x55dfdcc018d0 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f8323400640 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83140c6e20 tx=0x7f8314003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237a7b46ee7d5d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.355+0000 7f830f400640 1 -- 172.21.15.7:0/3667726520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140bde10 con 0x55dfdcc018d0 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.356+0000 7f830f400640 1 -- 172.21.15.7:0/3667726520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314005690 con 0x55dfdcc018d0 2024-04-20T21:36:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.356+0000 7f830f400640 1 -- 172.21.15.7:0/3667726520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831402de20 con 0x55dfdcc018d0 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- 172.21.15.7:0/3667726520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 msgr2=0x55dfdcc16190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f83140c6e20 tx=0x7f8314003840 comp rx=0 tx=0).stop 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- 172.21.15.7:0/3667726520 shutdown_connections 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 --2- 172.21.15.7:0/3667726520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc018d0 0x55dfdcc16190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- 172.21.15.7:0/3667726520 >> 172.21.15.7:0/3667726520 conn(0x55dfdcbfb6e0 msgr2=0x55dfdcbf9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- 172.21.15.7:0/3667726520 shutdown_connections 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- 172.21.15.7:0/3667726520 wait complete. 2024-04-20T21:36:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.357+0000 7f832cdd8740 1 -- start start 2024-04-20T21:36:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.358+0000 7f832cdd8740 1 Processor -- start 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.358+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc018d0 0x55dfdcc126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc12be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b2730 con 0x55dfdcc0a4b0 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b3ba0 con 0x55dfdcc018d0 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b2e00 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f832cdd8740 10 client.0 _ll_register_callbacks cb 0x7f83140efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f8323400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34358/0 (socket says 172.21.15.7:34358) 2024-04-20T21:36:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.359+0000 7f8323400640 1 -- 172.21.15.7:0/1815582950 learned_addr learned my addr 172.21.15.7:0/1815582950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.360+0000 7f8323e00640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc12be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.360+0000 7f8323400640 1 -- 172.21.15.7:0/1815582950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc018d0 msgr2=0x55dfdcc126a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.361+0000 7f8322a00640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc018d0 0x55dfdcc126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.361+0000 7f8323400640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc018d0 0x55dfdcc126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.362+0000 7f8323400640 1 -- 172.21.15.7:0/1815582950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 msgr2=0x55dfdcc12be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.362+0000 7f8323400640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc12be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.362+0000 7f8323400640 1 -- 172.21.15.7:0/1815582950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfdc9427d0 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.362+0000 7f8323e00640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc12be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.363+0000 7f8323400640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83140bac50 tx=0x7f83140efd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3b873609697d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.363+0000 7f830cc00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.363+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831402de20 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.364+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dfdcb6d8e0 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.364+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f83140bd9f0 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.365+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83140bde10 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.365+0000 7f832cdd8740 10 client.25921 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.365+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfdc9427d0 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.365+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dfdcb6d8e0 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.368+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f83140bd190 con 0x55dfdcbfa5c0 2024-04-20T21:36:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.368+0000 7f832cdd8740 10 client.25921 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.368+0000 7f832cdd8740 10 client.25921 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:36:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.369+0000 7f832cdd8740 20 client.25921 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.369+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc91c30 0x55dfdcc94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.369+0000 7f832cdd8740 4 client.25921 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.369+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfdcae95a0 con 0x55dfdcc91c30 2024-04-20T21:36:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.370+0000 7f8322a00640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc91c30 0x55dfdcc94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.373+0000 7f830cc00640 10 client.25921 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.373+0000 7f8322a00640 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc91c30 0x55dfdcc94020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d24885b7e5673760 server_cookie=d08ca7e942986c68 in_seq=0 out_seq=0 2024-04-20T21:36:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.373+0000 7f830cc00640 1 -- 172.21.15.7:0/1815582950 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfdcae95a0 con 0x55dfdcc91c30 2024-04-20T21:36:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.373+0000 7f830cc00640 10 client.25921 handle_command_reply: tid=0 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.373+0000 7f832cdd8740 1 client.25921 shutdown 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 msgr2=0x55dfdcc7bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f83140bac50 tx=0x7f83140efd90 comp rx=0 tx=0).stop 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 shutdown_connections 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55dfdcc91c30 0x55dfdcc94020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dfdcc018d0 0x55dfdcc126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f830cc00640 0 client.25921 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55dfdcc0a4b0 0x55dfdcc12be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 --2- 172.21.15.7:0/1815582950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dfdcbfa5c0 0x55dfdcc7bf60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 >> 172.21.15.7:0/1815582950 conn(0x55dfdcbfb6e0 msgr2=0x55dfdcbfc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 shutdown_connections 2024-04-20T21:36:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 1 -- 172.21.15.7:0/1815582950 wait complete. 2024-04-20T21:36:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.374+0000 7f832cdd8740 20 client.25921 trim_cache size 0 max 16384 2024-04-20T21:36:16.374 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 msgr2=0x7f83080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 0x7f83080a6260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f831c0125f0 tx=0x7f831c004a40 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 msgr2=0x7f832419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 0x7f832419ee00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f831402c210 tx=0x7f831402c7d0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 shutdown_connections 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83241062e0 0x7f83241a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f83241018e0 0x7f832419f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f83080a3da0 0x7f83080a6260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 --2- 172.21.15.7:0/2297212992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8324100fc0 0x7f832419ee00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.375+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 >> 172.21.15.7:0/2297212992 conn(0x7f8324078010 msgr2=0x7f8324103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.376+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 shutdown_connections 2024-04-20T21:36:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.376+0000 7f832a000640 1 -- 172.21.15.7:0/2297212992 wait complete. 2024-04-20T21:36:16.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:36:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.685+0000 7fcc4c200640 1 -- 172.21.15.7:0/2088535800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44102b50 msgr2=0x7fcc44106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.685+0000 7fcc4c200640 1 --2- 172.21.15.7:0/2088535800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44102b50 0x7fcc44106b90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009be0 tx=0x7fcc3002bef0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.686+0000 7fcc4c200640 1 -- 172.21.15.7:0/2088535800 shutdown_connections 2024-04-20T21:36:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.686+0000 7fcc4c200640 1 --2- 172.21.15.7:0/2088535800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc44107270 0x7fcc4410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.686+0000 7fcc4c200640 1 --2- 172.21.15.7:0/2088535800 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44102b50 0x7fcc44106b90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.686+0000 7fcc4c200640 1 --2- 172.21.15.7:0/2088535800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc44102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.686+0000 7fcc4c200640 1 -- 172.21.15.7:0/2088535800 >> 172.21.15.7:0/2088535800 conn(0x7fcc440fda20 msgr2=0x7fcc440ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.687+0000 7fcc4c200640 1 -- 172.21.15.7:0/2088535800 shutdown_connections 2024-04-20T21:36:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.688+0000 7fcc4c200640 1 -- 172.21.15.7:0/2088535800 wait complete. 2024-04-20T21:36:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.690+0000 7fcc4c200640 1 Processor -- start 2024-04-20T21:36:16.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.692+0000 7fcc4c200640 1 -- start start 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44107270 0x7fcc441a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc4419a910 0x7fcc4419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441123b0 con 0x7fcc4419a910 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112230 con 0x7fcc441021a0 2024-04-20T21:36:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.696+0000 7fcc4c200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44112530 con 0x7fcc44107270 2024-04-20T21:36:16.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.701+0000 7fcc48e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44107270 0x7fcc441a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.701+0000 7fcc49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.701+0000 7fcc49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46380/0 (socket says 172.21.15.7:46380) 2024-04-20T21:36:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.701+0000 7fcc49800640 1 -- 172.21.15.7:0/3786449468 learned_addr learned my addr 172.21.15.7:0/3786449468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.702+0000 7fcc49800640 1 -- 172.21.15.7:0/3786449468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44107270 msgr2=0x7fcc441a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.702+0000 7fcc49800640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44107270 0x7fcc441a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.703+0000 7fcc49800640 1 -- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc4419a910 msgr2=0x7fcc4419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.703+0000 7fcc49800640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc4419a910 0x7fcc4419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.703+0000 7fcc49800640 1 -- 172.21.15.7:0/3786449468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4419b430 con 0x7fcc441021a0 2024-04-20T21:36:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.707+0000 7fcc49800640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9c0 tx=0x7fcc3400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.707+0000 7fcc42000640 1 -- 172.21.15.7:0/3786449468 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400ccc0 con 0x7fcc441021a0 2024-04-20T21:36:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.710+0000 7fcc42000640 1 -- 172.21.15.7:0/3786449468 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc34005500 con 0x7fcc441021a0 2024-04-20T21:36:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.710+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4419b720 con 0x7fcc441021a0 2024-04-20T21:36:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.710+0000 7fcc42000640 1 -- 172.21.15.7:0/3786449468 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34010430 con 0x7fcc441021a0 2024-04-20T21:36:16.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.710+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc441a7970 con 0x7fcc441021a0 2024-04-20T21:36:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.731+0000 7fcc4ee11740 1 Processor -- start 2024-04-20T21:36:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.733+0000 7fcc4ee11740 1 -- start start 2024-04-20T21:36:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 0x561c5545aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545b0e0 0x561c55463780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561c553b7050 con 0x561c5545b0e0 2024-04-20T21:36:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c5535d490 con 0x561c5545a7c0 2024-04-20T21:36:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.734+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561c553b3010 con 0x561c55463cc0 2024-04-20T21:36:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.735+0000 7fcc48e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.735+0000 7fcc48e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34404/0 (socket says 172.21.15.7:34404) 2024-04-20T21:36:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.735+0000 7fcc4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 0x561c5545aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.735+0000 7fcc4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 0x561c5545aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46388/0 (socket says 172.21.15.7:46388) 2024-04-20T21:36:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.735+0000 7fcc48e00640 1 -- 172.21.15.7:0/3516633531 learned_addr learned my addr 172.21.15.7:0/3516633531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.736+0000 7fcc48e00640 1 -- 172.21.15.7:0/3516633531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 msgr2=0x561c5545aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.736+0000 7fcc48e00640 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 0x561c5545aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.736+0000 7fcc48e00640 1 -- 172.21.15.7:0/3516633531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545b0e0 msgr2=0x561c55463780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.736+0000 7fcc48e00640 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545b0e0 0x561c55463780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.736+0000 7fcc48e00640 1 -- 172.21.15.7:0/3516633531 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c55365070 con 0x561c55463cc0 2024-04-20T21:36:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.737+0000 7fcc48e00640 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc4406b570 tx=0x7fcc30033b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1937f9906ee370ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.737+0000 7fcc3aa00640 1 -- 172.21.15.7:0/3516633531 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30043030 con 0x561c55463cc0 2024-04-20T21:36:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.737+0000 7fcc3aa00640 1 -- 172.21.15.7:0/3516633531 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30033df0 con 0x561c55463cc0 2024-04-20T21:36:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.737+0000 7fcc3aa00640 1 -- 172.21.15.7:0/3516633531 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3002cab0 con 0x561c55463cc0 2024-04-20T21:36:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.738+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3516633531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 msgr2=0x561c554660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.738+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc4406b570 tx=0x7fcc30033b40 comp rx=0 tx=0).stop 2024-04-20T21:36:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.739+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3516633531 shutdown_connections 2024-04-20T21:36:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.739+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554660b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.739+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545b0e0 0x561c55463780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.739+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3516633531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c5545a7c0 0x561c5545aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.739+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3516633531 >> 172.21.15.7:0/3516633531 conn(0x561c55454ef0 msgr2=0x561c554552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.747+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3516633531 shutdown_connections 2024-04-20T21:36:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.748+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3516633531 wait complete. 2024-04-20T21:36:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.748+0000 7fcc4ee11740 1 -- start start 2024-04-20T21:36:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.749+0000 7fcc42000640 1 -- 172.21.15.7:0/3786449468 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcc340046c0 con 0x7fcc441021a0 2024-04-20T21:36:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.749+0000 7fcc4ee11740 1 Processor -- start 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c55480ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561c55477a60 con 0x561c5545a7c0 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561c55370860 con 0x561c554617f0 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561c55477be0 con 0x561c55463cc0 2024-04-20T21:36:16.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.752+0000 7fcc4ee11740 10 client.0 _ll_register_callbacks cb 0x561c5533efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.753+0000 7fcc42000640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 0x7fcc280a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.754+0000 7fcc4a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.754+0000 7fcc4a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57464/0 (socket says 172.21.15.7:57464) 2024-04-20T21:36:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.754+0000 7fcc4a200640 1 -- 172.21.15.7:0/3958092887 learned_addr learned my addr 172.21.15.7:0/3958092887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.755+0000 7fcc48e00640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c55480ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.755+0000 7fcc42000640 1 -- 172.21.15.7:0/3786449468 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc340d0b50 con 0x7fcc441021a0 2024-04-20T21:36:16.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 -- 172.21.15.7:0/3958092887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 msgr2=0x561c554612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 -- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 msgr2=0x561c55480ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c55480ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 -- 172.21.15.7:0/3958092887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c55374950 con 0x561c5545a7c0 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.756+0000 7fcc4a200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 0x7fcc280a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.757+0000 7fcc4a200640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00f260 tx=0x7fcc3c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375b00ad11a77516 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.757+0000 7fcc2f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:36:16.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.758+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c018040 con 0x561c5545a7c0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.758+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc30033df0 con 0x561c5545a7c0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.758+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc3c01fb20 con 0x561c5545a7c0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.759+0000 7fcc4a200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 0x7fcc280a6200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c01e000 tx=0x7fcc3c01f000 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.759+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc3c01fd20 con 0x561c5545a7c0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.760+0000 7fcc4ee11740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.760+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c55342820 con 0x561c5545a7c0 2024-04-20T21:36:16.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.760+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc30033df0 con 0x561c5545a7c0 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fcc3c01c070 con 0x561c5545a7c0 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 20 client.15813 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554851f0 0x561c55487690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 4 client.15813 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.761+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c553c4a70 con 0x561c554851f0 2024-04-20T21:36:16.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.762+0000 7fcc49800640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554851f0 0x561c55487690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.765+0000 7fcc2f400640 10 client.15813 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:16.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.765+0000 7fcc49800640 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554851f0 0x561c55487690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f28d089e4bcfa7 server_cookie=ab04f1cef72d8f73 in_seq=0 out_seq=0 2024-04-20T21:36:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.768+0000 7fcc2f400640 1 -- 172.21.15.7:0/3958092887 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c553c4a70 con 0x561c554851f0 2024-04-20T21:36:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.768+0000 7fcc2f400640 10 client.15813 handle_command_reply: tid=0 2024-04-20T21:36:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.768+0000 7fcc4ee11740 1 client.15813 shutdown 2024-04-20T21:36:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 msgr2=0x561c55460d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00f260 tx=0x7fcc3c00f720 comp rx=0 tx=0).stop 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 shutdown_connections 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c55480ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554851f0 0x561c55487690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c554612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/3958092887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c55460d70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 >> 172.21.15.7:0/3958092887 conn(0x561c55454ef0 msgr2=0x561c55456c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc2f400640 0 client.15813 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 shutdown_connections 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 1 -- 172.21.15.7:0/3958092887 wait complete. 2024-04-20T21:36:16.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.769+0000 7fcc4ee11740 20 client.15813 trim_cache size 0 max 16384 2024-04-20T21:36:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:16 smithi007 ceph-mon[16867]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 232 MiB/s wr, 82 op/s 2024-04-20T21:36:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:16 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/20473746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 Processor -- start 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 -- start start 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5548d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c55480920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44072040 con 0x561c5545a7c0 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4406f900 con 0x561c554617f0 2024-04-20T21:36:16.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.811+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4406d750 con 0x561c55463cc0 2024-04-20T21:36:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc4a200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c55480920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57472/0 (socket says 172.21.15.7:57472) 2024-04-20T21:36:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 -- 172.21.15.7:0/1820734789 learned_addr learned my addr 172.21.15.7:0/1820734789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 -- 172.21.15.7:0/1820734789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 msgr2=0x561c55480920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c55480920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 -- 172.21.15.7:0/1820734789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 msgr2=0x561c5548d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5548d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc49800640 1 -- 172.21.15.7:0/1820734789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c550c93f0 con 0x561c5545a7c0 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.812+0000 7fcc4a200640 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c55480920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.813+0000 7fcc49800640 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc34098f40 tx=0x7fcc340d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b18659b110d7650 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.813+0000 7fcc3a000640 1 -- 172.21.15.7:0/1820734789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340a0e90 con 0x561c5545a7c0 2024-04-20T21:36:16.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.813+0000 7fcc3a000640 1 -- 172.21.15.7:0/1820734789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c553c4a70 con 0x561c5545a7c0 2024-04-20T21:36:16.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.813+0000 7fcc3a000640 1 -- 172.21.15.7:0/1820734789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34099150 con 0x561c5545a7c0 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1820734789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 msgr2=0x561c5548cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc34098f40 tx=0x7fcc340d0040 comp rx=0 tx=0).stop 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1820734789 shutdown_connections 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5548d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c55480920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1820734789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545a7c0 0x561c5548cfa0 secure :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc34098f40 tx=0x7fcc340d0040 comp rx=0 tx=0).stop 2024-04-20T21:36:16.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.817+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1820734789 >> 172.21.15.7:0/1820734789 conn(0x561c55454ef0 msgr2=0x561c5545c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.818+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1820734789 shutdown_connections 2024-04-20T21:36:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.818+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1820734789 wait complete. 2024-04-20T21:36:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.818+0000 7fcc4ee11740 1 -- start start 2024-04-20T21:36:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.820+0000 7fcc4ee11740 1 Processor -- start 2024-04-20T21:36:16.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c5545d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545de20 0x561c5545e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4406ddc0 con 0x561c5545de20 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4406f230 con 0x561c554617f0 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4406e490 con 0x561c55463cc0 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc4ee11740 10 client.0 _ll_register_callbacks cb 0x7fcc3c00abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc49800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-04-20T21:36:16.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.821+0000 7fcc49800640 1 -- 172.21.15.7:0/1297617590 learned_addr learned my addr 172.21.15.7:0/1297617590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc48e00640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c5545d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc4a200640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545de20 0x561c5545e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc49800640 1 -- 172.21.15.7:0/1297617590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 msgr2=0x561c5545d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc49800640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c5545d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc49800640 1 -- 172.21.15.7:0/1297617590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545de20 msgr2=0x561c5545e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc49800640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545de20 0x561c5545e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.822+0000 7fcc49800640 1 -- 172.21.15.7:0/1297617590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c551937d0 con 0x561c554617f0 2024-04-20T21:36:16.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc49800640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc3409c700 tx=0x7fcc34009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603b834a45647c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:16.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc2f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:16.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc340a0e90 con 0x561c554617f0 2024-04-20T21:36:16.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcc340ce580 con 0x561c554617f0 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c553c4a70 con 0x561c554617f0 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.823+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc340a5750 con 0x561c554617f0 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.824+0000 7fcc4ee11740 10 client.25952 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.824+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c554ebb40 con 0x561c554617f0 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.824+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c553c4a70 con 0x561c554617f0 2024-04-20T21:36:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fcc340b1130 con 0x561c554617f0 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 10 client.25952 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 20 client.25952 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554ebf80 0x561c554ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 4 client.25952 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c553be8e0 con 0x561c554ebf80 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.825+0000 7fcc48e00640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554ebf80 0x561c554ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc48e00640 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554ebf80 0x561c554ee370 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92c0b9350ab4264 server_cookie=8b8dd3c1ea5f1f1 in_seq=0 out_seq=0 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc2f400640 10 client.25952 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc2f400640 1 -- 172.21.15.7:0/1297617590 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c553be8e0 con 0x561c554ebf80 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc2f400640 10 client.25952 handle_command_reply: tid=0 2024-04-20T21:36:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc4ee11740 1 client.25952 shutdown 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 msgr2=0x561c5547d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.826+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc3409c700 tx=0x7fcc34009f90 comp rx=0 tx=0).stop 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 shutdown_connections 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561c554ebf80 0x561c554ee370 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561c5545de20 0x561c5545e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561c55463cc0 0x561c5545d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 --2- 172.21.15.7:0/1297617590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561c554617f0 0x561c5547d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 >> 172.21.15.7:0/1297617590 conn(0x561c55454ef0 msgr2=0x561c55456320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc2f400640 0 client.25952 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 shutdown_connections 2024-04-20T21:36:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 1 -- 172.21.15.7:0/1297617590 wait complete. 2024-04-20T21:36:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.827+0000 7fcc4ee11740 20 client.25952 trim_cache size 0 max 16384 2024-04-20T21:36:16.827 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 msgr2=0x7fcc280a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 0x7fcc280a6200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c01e000 tx=0x7fcc3c01f000 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 msgr2=0x7fcc441a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9c0 tx=0x7fcc3400ee80 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 shutdown_connections 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc4419a910 0x7fcc4419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcc44107270 0x7fcc441a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcc280a3d40 0x7fcc280a6200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 --2- 172.21.15.7:0/3786449468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc441021a0 0x7fcc441a0710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 >> 172.21.15.7:0/3786449468 conn(0x7fcc440fda20 msgr2=0x7fcc440ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 shutdown_connections 2024-04-20T21:36:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:16.828+0000 7fcc4c200640 1 -- 172.21.15.7:0/3786449468 wait complete. 2024-04-20T21:36:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:16 smithi016 ceph-mon[21409]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 232 MiB/s wr, 82 op/s 2024-04-20T21:36:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:16 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/20473746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:16 smithi186 ceph-mon[22157]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 232 MiB/s wr, 82 op/s 2024-04-20T21:36:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:16 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/20473746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:17.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:17 smithi007 ceph-mon[16867]: pgmap v91: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 388 MiB/s wr, 125 op/s 2024-04-20T21:36:17.864 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:36:17.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:36:17.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:36:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:17 smithi016 ceph-mon[21409]: pgmap v91: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 388 MiB/s wr, 125 op/s 2024-04-20T21:36:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:17 smithi186 ceph-mon[22157]: pgmap v91: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 388 MiB/s wr, 125 op/s 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 -- 172.21.15.7:0/1540185690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 msgr2=0x7fb0b0076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 --2- 172.21.15.7:0/1540185690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b0076870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009930 tx=0x7fb0a002bc30 comp rx=0 tx=0).stop 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 -- 172.21.15.7:0/1540185690 shutdown_connections 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 --2- 172.21.15.7:0/1540185690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 0x7fb0b01128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 --2- 172.21.15.7:0/1540185690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 0x7fb0b01151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 --2- 172.21.15.7:0/1540185690 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b0076870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 -- 172.21.15.7:0/1540185690 >> 172.21.15.7:0/1540185690 conn(0x7fb0b00ffc10 msgr2=0x7fb0b0102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 -- 172.21.15.7:0/1540185690 shutdown_connections 2024-04-20T21:36:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.104+0000 7fb0b7200640 1 -- 172.21.15.7:0/1540185690 wait complete. 2024-04-20T21:36:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 Processor -- start 2024-04-20T21:36:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 -- start start 2024-04-20T21:36:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 0x7fb0b019f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 0x7fb0b01a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0118980 con 0x7fb0b0112de0 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0118800 con 0x7fb0b0076db0 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.105+0000 7fb0b7200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0118b00 con 0x7fb0b0076490 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34440/0 (socket says 172.21.15.7:34440) 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 -- 172.21.15.7:0/3816130515 learned_addr learned my addr 172.21.15.7:0/3816130515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0b5200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 0x7fb0b01a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0af400640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 0x7fb0b019f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 -- 172.21.15.7:0/3816130515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 msgr2=0x7fb0b019f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 0x7fb0b019f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 -- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 msgr2=0x7fb0b01a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 0x7fb0b01a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.106+0000 7fb0afe00640 1 -- 172.21.15.7:0/3816130515 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b01a3dd0 con 0x7fb0b0076490 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.107+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0000c00 tx=0x7fb0a00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.107+0000 7fb0acc00640 1 -- 172.21.15.7:0/3816130515 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0043020 con 0x7fb0b0076490 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.107+0000 7fb0acc00640 1 -- 172.21.15.7:0/3816130515 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a00035f0 con 0x7fb0b0076490 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.107+0000 7fb0acc00640 1 -- 172.21.15.7:0/3816130515 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a003e6b0 con 0x7fb0b0076490 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.107+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01a4060 con 0x7fb0b0076490 2024-04-20T21:36:18.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.108+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01ab950 con 0x7fb0b0076490 2024-04-20T21:36:18.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.111+0000 7fb0acc00640 1 -- 172.21.15.7:0/3816130515 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb0a0046020 con 0x7fb0b0076490 2024-04-20T21:36:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.112+0000 7fb0acc00640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 0x7fb0940a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0acc00640 1 -- 172.21.15.7:0/3816130515 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb0a00f2980 con 0x7fb0b0076490 2024-04-20T21:36:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 Processor -- start 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0af400640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 0x7fb0940a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 -- start start 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0da830 0x564bce0e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.113+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564bce03ed20 con 0x564bce0da830 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bcdfe53e0 con 0x564bce0e3410 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564bce03ace0 con 0x564bce0d9f10 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34450/0 (socket says 172.21.15.7:34450) 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0afe00640 1 -- 172.21.15.7:0/2333169045 learned_addr learned my addr 172.21.15.7:0/2333169045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0af400640 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0da830 0x564bce0e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 -- 172.21.15.7:0/2333169045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 msgr2=0x564bce0da2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 -- 172.21.15.7:0/2333169045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0da830 msgr2=0x564bce0e2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0da830 0x564bce0e2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0b5200640 1 -- 172.21.15.7:0/2333169045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bce004980 con 0x564bce0e3410 2024-04-20T21:36:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0afe00640 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.114+0000 7fb0af400640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 0x7fb0940a62a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005fd0 tx=0x7fb09c005850 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.115+0000 7fb0b5200640 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e5800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007d90 tx=0x7fb0a800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76dbef1ee72756b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.115+0000 7fb0a5600640 1 -- 172.21.15.7:0/2333169045 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003b70 con 0x564bce0e3410 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.115+0000 7fb0a5600640 1 -- 172.21.15.7:0/2333169045 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8003d10 con 0x564bce0e3410 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.115+0000 7fb0a5600640 1 -- 172.21.15.7:0/2333169045 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8010a70 con 0x564bce0e3410 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 -- 172.21.15.7:0/2333169045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 msgr2=0x564bce0e5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e5800 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007d90 tx=0x7fb0a800f790 comp rx=0 tx=0).stop 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 -- 172.21.15.7:0/2333169045 shutdown_connections 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e5800 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0da830 0x564bce0e2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/2333169045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0da2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 -- 172.21.15.7:0/2333169045 >> 172.21.15.7:0/2333169045 conn(0x564bce0d4640 msgr2=0x564bce0d4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 -- 172.21.15.7:0/2333169045 shutdown_connections 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.116+0000 7fb0b9f73740 1 -- 172.21.15.7:0/2333169045 wait complete. 2024-04-20T21:36:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 -- start start 2024-04-20T21:36:18.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 Processor -- start 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0d9f10 0x564bce0e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564bce0e6200 con 0x564bce0d9f10 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564bcdff87a0 con 0x564bce0e3410 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564bce0e6380 con 0x564bce0da830 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.117+0000 7fb0b9f73740 10 client.0 _ll_register_callbacks cb 0x564bcdffab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46456/0 (socket says 172.21.15.7:46456) 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34466/0 (socket says 172.21.15.7:34466) 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0b5200640 1 -- 172.21.15.7:0/964372736 learned_addr learned my addr 172.21.15.7:0/964372736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0afe00640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0d9f10 0x564bce0e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 -- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 msgr2=0x564bce0e0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 -- 172.21.15.7:0/964372736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0d9f10 msgr2=0x564bce0e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0d9f10 0x564bce0e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.118+0000 7fb0af400640 1 -- 172.21.15.7:0/964372736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bcde197e0 con 0x564bce0da830 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.119+0000 7fb0af400640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00bfd0 tx=0x7fb09c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e408edc7fc45cfdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.119+0000 7fb09a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.119+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c01e830 con 0x564bce0da830 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.119+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb09c01e9d0 con 0x564bce0da830 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.119+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8003d10 con 0x564bce0da830 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb09c01ebd0 con 0x564bce0da830 2024-04-20T21:36:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 10 client.25948 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564bcde79f70 con 0x564bce0da830 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8003d10 con 0x564bce0da830 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fb09c02f3d0 con 0x564bce0da830 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 10 client.25948 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 20 client.25948 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce0f37e0 0x564bce0f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 4 client.25948 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.120+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bce04c740 con 0x564bce0f37e0 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.121+0000 7fb0afe00640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce0f37e0 0x564bce0f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.121+0000 7fb0afe00640 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce0f37e0 0x564bce0f5c30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9905e1773e15bb8d server_cookie=762ed846223817b4 in_seq=0 out_seq=0 2024-04-20T21:36:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.121+0000 7fb09a000640 10 client.25948 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.122+0000 7fb09a000640 1 -- 172.21.15.7:0/964372736 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564bce04c740 con 0x564bce0f37e0 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.122+0000 7fb09a000640 10 client.25948 handle_command_reply: tid=0 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.122+0000 7fb0b9f73740 1 client.25948 shutdown 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.122+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 msgr2=0x564bce0e0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.122+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00bfd0 tx=0x7fb09c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 shutdown_connections 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0e3410 0x564bce0e0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce0f37e0 0x564bce0f5c30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb09a000640 0 client.25948 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0da830 0x564bce0e0950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/964372736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0d9f10 0x564bce0e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 >> 172.21.15.7:0/964372736 conn(0x564bce0d4640 msgr2=0x564bce0d6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 shutdown_connections 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 1 -- 172.21.15.7:0/964372736 wait complete. 2024-04-20T21:36:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.123+0000 7fb0b9f73740 20 client.25948 trim_cache size 0 max 16384 2024-04-20T21:36:18.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 Processor -- start 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 -- start start 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0fb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.156+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b69a0 con 0x564bce0e3410 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b4260 con 0x564bce0da830 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b20b0 con 0x564bce0d9f10 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46472/0 (socket says 172.21.15.7:46472) 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0af400640 1 -- 172.21.15.7:0/741595897 learned_addr learned my addr 172.21.15.7:0/741595897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.157+0000 7fb0b5200640 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 -- 172.21.15.7:0/741595897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 msgr2=0x564bce0e2b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0e2b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 -- 172.21.15.7:0/741595897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 msgr2=0x564bce0fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 -- 172.21.15.7:0/741595897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bcdd11170 con 0x564bce0da830 2024-04-20T21:36:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0b5200640 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.158+0000 7fb0af400640 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f1bfb77521bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.159+0000 7fb0a4c00640 1 -- 172.21.15.7:0/741595897 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c035040 con 0x564bce0da830 2024-04-20T21:36:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.159+0000 7fb0a4c00640 1 -- 172.21.15.7:0/741595897 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c02fa30 con 0x564bce0da830 2024-04-20T21:36:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.159+0000 7fb0a4c00640 1 -- 172.21.15.7:0/741595897 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0053f0 con 0x564bce0da830 2024-04-20T21:36:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.160+0000 7fb0b9f73740 1 -- 172.21.15.7:0/741595897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 msgr2=0x564bce0ef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.160+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb09c000c00 tx=0x7fb09c011ca0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 -- 172.21.15.7:0/741595897 shutdown_connections 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0ef0f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/741595897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 -- 172.21.15.7:0/741595897 >> 172.21.15.7:0/741595897 conn(0x564bce0d4640 msgr2=0x564bce0e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 -- 172.21.15.7:0/741595897 shutdown_connections 2024-04-20T21:36:18.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.161+0000 7fb0b9f73740 1 -- 172.21.15.7:0/741595897 wait complete. 2024-04-20T21:36:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.163+0000 7fb0b9f73740 1 -- start start 2024-04-20T21:36:18.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.164+0000 7fb0b9f73740 1 Processor -- start 2024-04-20T21:36:18.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b2720 con 0x564bce0e3410 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b3b90 con 0x564bce0da830 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b2df0 con 0x564bce0d9f10 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0b9f73740 10 client.0 _ll_register_callbacks cb 0x7fb09c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:18.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0af400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0afe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34476/0 (socket says 172.21.15.7:34476) 2024-04-20T21:36:18.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.167+0000 7fb0afe00640 1 -- 172.21.15.7:0/3010691380 learned_addr learned my addr 172.21.15.7:0/3010691380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.168+0000 7fb0afe00640 1 -- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 msgr2=0x564bce0e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.168+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.168+0000 7fb0afe00640 1 -- 172.21.15.7:0/3010691380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 msgr2=0x564bce0e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0b5200640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0e7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0afe00640 1 -- 172.21.15.7:0/3010691380 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bce0e84b0 con 0x564bce0d9f10 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0af400640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0b5200640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0afe00640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00df0d0 tx=0x7fb0a00df040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfe613bad4b45be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb09a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00d7070 con 0x564bce0d9f10 2024-04-20T21:36:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb0a00037a0 con 0x564bce0d9f10 2024-04-20T21:36:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.169+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09c02fa30 con 0x564bce0d9f10 2024-04-20T21:36:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.170+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0a00a4050 con 0x564bce0d9f10 2024-04-20T21:36:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.170+0000 7fb0b9f73740 10 client.25951 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.170+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564bce15ab50 con 0x564bce0d9f10 2024-04-20T21:36:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.170+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c02fa30 con 0x564bce0d9f10 2024-04-20T21:36:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.171+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fb0a00dc020 con 0x564bce0d9f10 2024-04-20T21:36:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.171+0000 7fb0b9f73740 10 client.25951 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.171+0000 7fb0b9f73740 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.171+0000 7fb0b9f73740 20 client.25951 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.171+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce15b300 0x564bce15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.172+0000 7fb0b9f73740 4 client.25951 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.172+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564bce0465b0 con 0x564bce15b300 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.172+0000 7fb0af400640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce15b300 0x564bce15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.173+0000 7fb09a000640 10 client.25951 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.173+0000 7fb0af400640 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce15b300 0x564bce15d720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6faf0df8dcc22fc8 server_cookie=c26443b97fa79d0 in_seq=0 out_seq=0 2024-04-20T21:36:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb09a000640 1 -- 172.21.15.7:0/3010691380 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564bce0465b0 con 0x564bce15b300 2024-04-20T21:36:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb09a000640 10 client.25951 handle_command_reply: tid=0 2024-04-20T21:36:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 client.25951 shutdown 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 msgr2=0x564bce0eb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00df0d0 tx=0x7fb0a00df040 comp rx=0 tx=0).stop 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 shutdown_connections 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564bce0da830 0x564bce0e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564bce15b300 0x564bce15d720 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564bce0e3410 0x564bce0e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 --2- 172.21.15.7:0/3010691380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564bce0d9f10 0x564bce0eb6b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.175+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 >> 172.21.15.7:0/3010691380 conn(0x564bce0d4640 msgr2=0x564bce0d2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.176+0000 7fb09a000640 0 client.25951 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.176+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 shutdown_connections 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.176+0000 7fb0b9f73740 1 -- 172.21.15.7:0/3010691380 wait complete. 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.176+0000 7fb0b9f73740 20 client.25951 trim_cache size 0 max 16384 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "ac4a3189-66c0-4183-ae7e-7ef361d7b505", 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:36:18.175 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 msgr2=0x7fb0940a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 0x7fb0940a62a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb09c005fd0 tx=0x7fb09c005850 comp rx=0 tx=0).stop 2024-04-20T21:36:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 msgr2=0x7fb0b019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0000c00 tx=0x7fb0a00042f0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 shutdown_connections 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb0940a3de0 0x7fb0940a62a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b0076db0 0x7fb0b019f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0b0112de0 0x7fb0b01a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 --2- 172.21.15.7:0/3816130515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0b0076490 0x7fb0b019ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.177+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 >> 172.21.15.7:0/3816130515 conn(0x7fb0b00ffc10 msgr2=0x7fb0b0102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.178+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 shutdown_connections 2024-04-20T21:36:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.178+0000 7fb0b7200640 1 -- 172.21.15.7:0/3816130515 wait complete. 2024-04-20T21:36:18.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:18.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:18.342 INFO:tasks.workunit.client.0.smithi007.stdout:Syncing()...6 sec 2024-04-20T21:36:18.344 INFO:tasks.workunit.client.0.smithi007.stdout:Starting Actual Benchmark At: Sat Apr 20 21:36:18 2024 2024-04-20T21:36:18.344 INFO:tasks.workunit.client.0.smithi007.stdout: 2024-04-20T21:36:18.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:18.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:18.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.504 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:18.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:18.505 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:18 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.558+0000 7f2e62e00640 1 -- 172.21.15.7:0/30091304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1104e0 msgr2=0x7f2e5c1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.558+0000 7f2e62e00640 1 --2- 172.21.15.7:0/30091304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1104e0 0x7f2e5c1108c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a910 tx=0x7f2e4803cca0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.565+0000 7f2e62e00640 1 -- 172.21.15.7:0/30091304 shutdown_connections 2024-04-20T21:36:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.565+0000 7f2e62e00640 1 --2- 172.21.15.7:0/30091304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c11d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.565+0000 7f2e62e00640 1 --2- 172.21.15.7:0/30091304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10e040 0x7f2e5c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.565+0000 7f2e62e00640 1 --2- 172.21.15.7:0/30091304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c1104e0 0x7f2e5c1108c0 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a910 tx=0x7f2e4803cca0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.565+0000 7f2e62e00640 1 -- 172.21.15.7:0/30091304 >> 172.21.15.7:0/30091304 conn(0x7f2e5c06ce70 msgr2=0x7f2e5c06d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.577+0000 7f2e62e00640 1 -- 172.21.15.7:0/30091304 shutdown_connections 2024-04-20T21:36:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.579+0000 7f2e62e00640 1 -- 172.21.15.7:0/30091304 wait complete. 2024-04-20T21:36:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.581+0000 7f2e62e00640 1 Processor -- start 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.585+0000 7f2e62e00640 1 -- start start 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.585+0000 7f2e62e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.585+0000 7f2e62e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.587+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.587+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34480/0 (socket says 172.21.15.7:34480) 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e62e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c119de0 0x7f2e5c1ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e61000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e61000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57558/0 (socket says 172.21.15.7:57558) 2024-04-20T21:36:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e62e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1139a0 con 0x7f2e5c10eb60 2024-04-20T21:36:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e62e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c113820 con 0x7f2e5c119de0 2024-04-20T21:36:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e62e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c113b20 con 0x7f2e5c10e040 2024-04-20T21:36:18.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.590+0000 7f2e61000640 1 -- 172.21.15.7:0/1233809641 learned_addr learned my addr 172.21.15.7:0/1233809641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.591+0000 7f2e61000640 1 -- 172.21.15.7:0/1233809641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 msgr2=0x7f2e5c11d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.591+0000 7f2e61000640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.591+0000 7f2e61000640 1 -- 172.21.15.7:0/1233809641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c119de0 msgr2=0x7f2e5c1ab360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.591+0000 7f2e61000640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c119de0 0x7f2e5c1ab360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.591+0000 7f2e61000640 1 -- 172.21.15.7:0/1233809641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c1ab8a0 con 0x7f2e5c10eb60 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.592+0000 7f2e61000640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b450 tx=0x7f2e5000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.593+0000 7f2e61a00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.595+0000 7f2e5a000640 1 -- 172.21.15.7:0/1233809641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50015020 con 0x7f2e5c10eb60 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.595+0000 7f2e5a000640 1 -- 172.21.15.7:0/1233809641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e50011d70 con 0x7f2e5c10eb60 2024-04-20T21:36:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.595+0000 7f2e5a000640 1 -- 172.21.15.7:0/1233809641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5000f340 con 0x7f2e5c10eb60 2024-04-20T21:36:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.599+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c1aba70 con 0x7f2e5c10eb60 2024-04-20T21:36:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.599+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1abe80 con 0x7f2e5c10eb60 2024-04-20T21:36:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.611+0000 7f2e6702d740 1 Processor -- start 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.615+0000 7f2e6702d740 1 -- start start 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.615+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.615+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e62400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e62400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57562/0 (socket says 172.21.15.7:57562) 2024-04-20T21:36:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e62400640 1 -- 172.21.15.7:0/2957298691 learned_addr learned my addr 172.21.15.7:0/2957298691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e6702d740 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a97305dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a972fb8150 con 0x55a9730523b0 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a972f5e370 con 0x55a97305b8b0 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.616+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a972fb4110 con 0x55a973052cd0 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e61a00640 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e62400640 1 -- 172.21.15.7:0/2957298691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 msgr2=0x55a97305b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e62400640 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e62400640 1 -- 172.21.15.7:0/2957298691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 msgr2=0x55a97305dca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e62400640 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a97305dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.617+0000 7f2e62400640 1 -- 172.21.15.7:0/2957298691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a972d947d0 con 0x55a9730523b0 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.618+0000 7f2e5a000640 1 -- 172.21.15.7:0/1233809641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2e50018020 con 0x7f2e5c10eb60 2024-04-20T21:36:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.618+0000 7f2e62400640 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009f60 tx=0x7f2e54009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2f3a157c6089b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.619+0000 7f2e4ea00640 1 -- 172.21.15.7:0/2957298691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400bca0 con 0x55a9730523b0 2024-04-20T21:36:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.619+0000 7f2e4ea00640 1 -- 172.21.15.7:0/2957298691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54017070 con 0x55a9730523b0 2024-04-20T21:36:18.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.619+0000 7f2e4ea00640 1 -- 172.21.15.7:0/2957298691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e540117f0 con 0x55a9730523b0 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.620+0000 7f2e5a000640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 0x7f2e440c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.623+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 msgr2=0x55a973052790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.624+0000 7f2e6702d740 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009f60 tx=0x7f2e54009f90 comp rx=0 tx=0).stop 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.628+0000 7f2e61a00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 0x7f2e440c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.630+0000 7f2e61a00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 0x7f2e440c8330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a860 tx=0x7f2e48005fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.630+0000 7f2e5a000640 1 -- 172.21.15.7:0/1233809641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e500d02e0 con 0x7f2e5c10eb60 2024-04-20T21:36:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.634+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 shutdown_connections 2024-04-20T21:36:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.634+0000 7f2e6702d740 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a97305dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.634+0000 7f2e6702d740 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.634+0000 7f2e6702d740 1 --2- 172.21.15.7:0/2957298691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a973052790 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.634+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 >> 172.21.15.7:0/2957298691 conn(0x55a97304cae0 msgr2=0x55a97304cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.637+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 shutdown_connections 2024-04-20T21:36:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.639+0000 7f2e6702d740 1 -- 172.21.15.7:0/2957298691 wait complete. 2024-04-20T21:36:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.639+0000 7f2e6702d740 1 -- start start 2024-04-20T21:36:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.640+0000 7f2e6702d740 1 Processor -- start 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a973058f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a97305e6a0 con 0x55a973052cd0 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a972f71740 con 0x55a97305b8b0 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a97305e820 con 0x55a9730523b0 2024-04-20T21:36:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.643+0000 7f2e6702d740 10 client.0 _ll_register_callbacks cb 0x55a972f6cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:34500/0 (socket says 172.21.15.7:34500) 2024-04-20T21:36:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 -- 172.21.15.7:0/4237391213 learned_addr learned my addr 172.21.15.7:0/4237391213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61000640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a973058f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e62400640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 -- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 msgr2=0x55a973059440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 -- 172.21.15.7:0/4237391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 msgr2=0x55a973058f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a973058f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.644+0000 7f2e61a00640 1 -- 172.21.15.7:0/4237391213 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a972cca3f0 con 0x55a9730523b0 2024-04-20T21:36:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.645+0000 7f2e61000640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a973058f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.650+0000 7f2e61a00640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a910 tx=0x7f2e4803cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37a1df0c69ee9b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.650+0000 7f2e43400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:18.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.650+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4805e020 con 0x55a9730523b0 2024-04-20T21:36:18.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.650+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e4803dc60 con 0x55a9730523b0 2024-04-20T21:36:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.654+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e54017070 con 0x55a9730523b0 2024-04-20T21:36:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.655+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e4803de60 con 0x55a9730523b0 2024-04-20T21:36:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.656+0000 7f2e6702d740 10 client.25957 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.656+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a972c8a0f0 con 0x55a9730523b0 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.656+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54017070 con 0x55a9730523b0 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f2e48003aa0 con 0x55a9730523b0 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 10 client.25957 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 20 client.25957 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a97306bdb0 0x55a97306e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 4 client.25957 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.657+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a972fc5b70 con 0x55a97306bdb0 2024-04-20T21:36:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.662+0000 7f2e61000640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a97306bdb0 0x55a97306e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.662+0000 7f2e61000640 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a97306bdb0 0x55a97306e1c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab5166d62d29f8c7 server_cookie=d25a49200187eb82 in_seq=0 out_seq=0 2024-04-20T21:36:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.662+0000 7f2e43400640 10 client.25957 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.663+0000 7f2e43400640 1 -- 172.21.15.7:0/4237391213 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a972fc5b70 con 0x55a97306bdb0 2024-04-20T21:36:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.663+0000 7f2e43400640 10 client.25957 handle_command_reply: tid=0 2024-04-20T21:36:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.663+0000 7f2e6702d740 1 client.25957 shutdown 2024-04-20T21:36:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.667+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 msgr2=0x55a9730589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.667+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a910 tx=0x7f2e4803cca0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 shutdown_connections 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a97306bdb0 0x55a97306e1c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a973058f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 --2- 172.21.15.7:0/4237391213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a9730589c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 >> 172.21.15.7:0/4237391213 conn(0x55a97304cae0 msgr2=0x55a97304e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.671+0000 7f2e43400640 0 client.25957 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.678+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 shutdown_connections 2024-04-20T21:36:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.684+0000 7f2e6702d740 1 -- 172.21.15.7:0/4237391213 wait complete. 2024-04-20T21:36:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.689+0000 7f2e6702d740 20 client.25957 trim_cache size 0 max 16384 2024-04-20T21:36:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 Processor -- start 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 -- start start 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a97305b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07f1f0 con 0x55a973052cd0 2024-04-20T21:36:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07cab0 con 0x55a97305b8b0 2024-04-20T21:36:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.798+0000 7f2e6702d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07a900 con 0x55a9730523b0 2024-04-20T21:36:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.799+0000 7f2e61000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.799+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a97305b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57574/0 (socket says 172.21.15.7:57574) 2024-04-20T21:36:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 -- 172.21.15.7:0/974118180 learned_addr learned my addr 172.21.15.7:0/974118180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 -- 172.21.15.7:0/974118180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 msgr2=0x55a97305b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a97305b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 -- 172.21.15.7:0/974118180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 msgr2=0x55a973059260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.801+0000 7f2e61000640 1 -- 172.21.15.7:0/974118180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97306e840 con 0x55a973052cd0 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.802+0000 7f2e61000640 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009fd0 tx=0x7f2e50098670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fccf96e74e6047d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.802+0000 7f2e4e000640 1 -- 172.21.15.7:0/974118180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500cf040 con 0x55a973052cd0 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.802+0000 7f2e4e000640 1 -- 172.21.15.7:0/974118180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e5009a420 con 0x55a973052cd0 2024-04-20T21:36:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.802+0000 7f2e4e000640 1 -- 172.21.15.7:0/974118180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5009ba30 con 0x55a973052cd0 2024-04-20T21:36:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.803+0000 7f2e6702d740 1 -- 172.21.15.7:0/974118180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 msgr2=0x55a9730572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.803+0000 7f2e6702d740 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009fd0 tx=0x7f2e50098670 comp rx=0 tx=0).stop 2024-04-20T21:36:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.808+0000 7f2e6702d740 1 -- 172.21.15.7:0/974118180 shutdown_connections 2024-04-20T21:36:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.808+0000 7f2e6702d740 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973059260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.808+0000 7f2e6702d740 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a973052cd0 0x55a9730572d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.808+0000 7f2e6702d740 1 --2- 172.21.15.7:0/974118180 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a9730523b0 0x55a97305b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.808+0000 7f2e6702d740 1 -- 172.21.15.7:0/974118180 >> 172.21.15.7:0/974118180 conn(0x55a97304cae0 msgr2=0x55a97304dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.816+0000 7f2e6702d740 1 -- 172.21.15.7:0/974118180 shutdown_connections 2024-04-20T21:36:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.816+0000 7f2e6702d740 1 -- 172.21.15.7:0/974118180 wait complete. 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.816+0000 7f2e6702d740 1 -- start start 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.817+0000 7f2e6702d740 1 Processor -- start 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.820+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.821+0000 7f2e6702d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.821+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.821+0000 7f2e61a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57586/0 (socket says 172.21.15.7:57586) 2024-04-20T21:36:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.821+0000 7f2e61a00640 1 -- 172.21.15.7:0/3744628939 learned_addr learned my addr 172.21.15.7:0/3744628939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.821+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973060190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.822+0000 7f2e62400640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973060190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.822+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07af70 con 0x55a9730523b0 2024-04-20T21:36:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.822+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07c3e0 con 0x55a97305b8b0 2024-04-20T21:36:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.822+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c07b640 con 0x55a973052cd0 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 -- 172.21.15.7:0/3744628939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 msgr2=0x55a97305fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 -- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 msgr2=0x55a973060190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973060190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9730d1550 con 0x55a9730523b0 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.823+0000 7f2e61a00640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e48003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee724ff96c1ce7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.824+0000 7f2e43400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:36:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.824+0000 7f2e6702d740 10 client.0 _ll_register_callbacks cb 0x7f2e480047e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.824+0000 7f2e43400640 1 -- 172.21.15.7:0/3744628939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48070020 con 0x55a9730523b0 2024-04-20T21:36:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.824+0000 7f2e43400640 1 -- 172.21.15.7:0/3744628939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2e480051f0 con 0x55a9730523b0 2024-04-20T21:36:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.831+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5009a420 con 0x55a9730523b0 2024-04-20T21:36:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.835+0000 7f2e43400640 1 -- 172.21.15.7:0/3744628939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2e4805e040 con 0x55a9730523b0 2024-04-20T21:36:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 10 client.15843 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55a9730d26b0 con 0x55a9730523b0 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e43400640 1 -- 172.21.15.7:0/3744628939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f2e480053f0 con 0x55a9730523b0 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 10 client.15843 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 20 client.15843 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.836+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a9730d2cf0 0x55a9730d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.837+0000 7f2e6702d740 4 client.15843 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.837+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a972fbf9e0 con 0x55a9730d2cf0 2024-04-20T21:36:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.837+0000 7f2e61000640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a9730d2cf0 0x55a9730d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.838+0000 7f2e43400640 10 client.15843 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.838+0000 7f2e61000640 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a9730d2cf0 0x55a9730d51b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197e1df367c89eb0 server_cookie=116c5c6f7ca850 in_seq=0 out_seq=0 2024-04-20T21:36:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.838+0000 7f2e43400640 1 -- 172.21.15.7:0/3744628939 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a972fbf9e0 con 0x55a9730d2cf0 2024-04-20T21:36:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.838+0000 7f2e43400640 10 client.15843 handle_command_reply: tid=0 2024-04-20T21:36:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.838+0000 7f2e6702d740 1 client.15843 shutdown 2024-04-20T21:36:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.841+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 msgr2=0x55a97305f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.841+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2e48000c00 tx=0x7f2e48003ad0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 shutdown_connections 2024-04-20T21:36:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a97305b8b0 0x55a973060190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a9730d2cf0 0x55a9730d51b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a973052cd0 0x55a97305fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 --2- 172.21.15.7:0/3744628939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9730523b0 0x55a97305f710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.850+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 >> 172.21.15.7:0/3744628939 conn(0x55a97304cae0 msgr2=0x55a97304e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.853+0000 7f2e43400640 0 client.15843 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.856+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 shutdown_connections 2024-04-20T21:36:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.857+0000 7f2e6702d740 1 -- 172.21.15.7:0/3744628939 wait complete. 2024-04-20T21:36:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.859+0000 7f2e6702d740 20 client.15843 trim_cache size 0 max 16384 2024-04-20T21:36:18.864 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:36:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:36:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: "ac4a3189-66c0-4183-ae7e-7ef361d7b505": { 2024-04-20T21:36:18.864 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:36:18.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "ac4a3189-66c0-4183-ae7e-7ef361d7b505", 2024-04-20T21:36:18.865 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:36:18.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:36:18.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:36:18.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 msgr2=0x7f2e440c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 0x7f2e440c8330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801a860 tx=0x7f2e48005fd0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 msgr2=0x7f2e5c119790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b450 tx=0x7f2e5000b910 comp rx=0 tx=0).stop 2024-04-20T21:36:18.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 shutdown_connections 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2e440c5e70 0x7f2e440c8330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e5c119de0 0x7f2e5c1ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e5c10eb60 0x7f2e5c119790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 --2- 172.21.15.7:0/1233809641 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2e5c10e040 0x7f2e5c11d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.878+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 >> 172.21.15.7:0/1233809641 conn(0x7f2e5c06ce70 msgr2=0x7f2e5c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.881+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 shutdown_connections 2024-04-20T21:36:18.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:18.887+0000 7f2e62e00640 1 -- 172.21.15.7:0/1233809641 wait complete. 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:18.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:18 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:18.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:18.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:18 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:18.999 INFO:tasks.cephfs.filesystem:scrub status for tag:ac4a3189-66c0-4183-ae7e-7ef361d7b505 - {'path': '/', 'tag': 'ac4a3189-66c0-4183-ae7e-7ef361d7b505', 'options': 'recursive,force'} 2024-04-20T21:36:19.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:19 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:36:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:19 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:19 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:19 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:19 smithi007 ceph-mon[16867]: pgmap v92: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:19 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:36:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:19 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:19 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:19 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:19 smithi016 ceph-mon[21409]: pgmap v92: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:19 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:36:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:19 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:19 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:19.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:19 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:19.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:19 smithi186 ceph-mon[22157]: pgmap v92: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:22.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:22 smithi016 ceph-mon[21409]: pgmap v93: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:22.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:22 smithi186 ceph-mon[22157]: pgmap v93: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:22.485 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:22 smithi007 ceph-mon[16867]: pgmap v93: 97 pgs: 97 active+clean; 12 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 289 MiB/s wr, 86 op/s 2024-04-20T21:36:24.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:24 smithi016 ceph-mon[21409]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 351 MiB/s wr, 4.43k op/s 2024-04-20T21:36:24.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:24 smithi186 ceph-mon[22157]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 351 MiB/s wr, 4.43k op/s 2024-04-20T21:36:24.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:24 smithi007 ceph-mon[16867]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 351 MiB/s wr, 4.43k op/s 2024-04-20T21:36:26.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:26 smithi016 ceph-mon[21409]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 217 MiB/s wr, 4.38k op/s 2024-04-20T21:36:26.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:26 smithi186 ceph-mon[22157]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 217 MiB/s wr, 4.38k op/s 2024-04-20T21:36:26.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:26 smithi007 ceph-mon[16867]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 217 MiB/s wr, 4.38k op/s 2024-04-20T21:36:27.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:27 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:36:27.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:27 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:36:27.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:27 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:36:28.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:28 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:36:28.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:28 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:36:28.466 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:28 smithi186 ceph-mon[22157]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 248 MiB/s wr, 8.91k op/s 2024-04-20T21:36:28.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:28 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:36:28.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:28 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:36:28.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:28 smithi007 ceph-mon[16867]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 248 MiB/s wr, 8.91k op/s 2024-04-20T21:36:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:28 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:36:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:28 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:36:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:28 smithi016 ceph-mon[21409]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 248 MiB/s wr, 8.91k op/s 2024-04-20T21:36:30.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:30 smithi007 ceph-mon[16867]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:30 smithi016 ceph-mon[21409]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:30.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:30 smithi186 ceph-mon[22157]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:32.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:32 smithi007 ceph-mon[16867]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:32 smithi016 ceph-mon[21409]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:32 smithi186 ceph-mon[22157]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 116 MiB/s wr, 12.57k op/s 2024-04-20T21:36:34.517 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:34 smithi186 ceph-mon[22157]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 176 MiB/s wr, 21.10k op/s 2024-04-20T21:36:34.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:34 smithi007 ceph-mon[16867]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 176 MiB/s wr, 21.10k op/s 2024-04-20T21:36:34.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:34 smithi016 ceph-mon[21409]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 176 MiB/s wr, 21.10k op/s 2024-04-20T21:36:35.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:35 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:35.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:35 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:35.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:35 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:36:36.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:36 smithi007 ceph-mon[16867]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 122 MiB/s wr, 17.81k op/s 2024-04-20T21:36:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:36 smithi016 ceph-mon[21409]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 122 MiB/s wr, 17.81k op/s 2024-04-20T21:36:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:36 smithi186 ceph-mon[22157]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 122 MiB/s wr, 17.81k op/s 2024-04-20T21:36:37.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:37 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:37.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:37 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:37.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:37 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:36:37.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:37 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:37.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:37 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:37.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:37 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:36:37.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:37 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:36:37.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:37 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:36:37.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:37 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:36:38.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:38 smithi007 ceph-mon[16867]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 127 MiB/s wr, 18.57k op/s 2024-04-20T21:36:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:38 smithi016 ceph-mon[21409]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 127 MiB/s wr, 18.57k op/s 2024-04-20T21:36:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:38 smithi186 ceph-mon[22157]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 127 MiB/s wr, 18.57k op/s 2024-04-20T21:36:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:40 smithi016 ceph-mon[21409]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 177 MiB/s wr, 25.76k op/s 2024-04-20T21:36:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:40 smithi186 ceph-mon[22157]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 177 MiB/s wr, 25.76k op/s 2024-04-20T21:36:40.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:40 smithi007 ceph-mon[16867]: pgmap v102: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 177 MiB/s wr, 25.76k op/s 2024-04-20T21:36:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:42 smithi016 ceph-mon[21409]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 154 MiB/s wr, 22.06k op/s 2024-04-20T21:36:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:42 smithi186 ceph-mon[22157]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 154 MiB/s wr, 22.06k op/s 2024-04-20T21:36:42.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:42 smithi007 ceph-mon[16867]: pgmap v103: 97 pgs: 97 active+clean; 15 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 154 MiB/s wr, 22.06k op/s 2024-04-20T21:36:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:44 smithi016 ceph-mon[21409]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 200 MiB/s wr, 28.93k op/s 2024-04-20T21:36:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:44 smithi186 ceph-mon[22157]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 200 MiB/s wr, 28.93k op/s 2024-04-20T21:36:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:44 smithi007 ceph-mon[16867]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 200 MiB/s wr, 28.93k op/s 2024-04-20T21:36:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:46 smithi016 ceph-mon[21409]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 158 MiB/s wr, 23.09k op/s 2024-04-20T21:36:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:46 smithi186 ceph-mon[22157]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 158 MiB/s wr, 23.09k op/s 2024-04-20T21:36:46.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:46 smithi007 ceph-mon[16867]: pgmap v105: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 158 MiB/s wr, 23.09k op/s 2024-04-20T21:36:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:48 smithi016 ceph-mon[21409]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 152 MiB/s wr, 22.23k op/s 2024-04-20T21:36:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:48 smithi186 ceph-mon[22157]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 152 MiB/s wr, 22.23k op/s 2024-04-20T21:36:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:48 smithi007 ceph-mon[16867]: pgmap v106: 97 pgs: 97 active+clean; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 152 MiB/s wr, 22.23k op/s 2024-04-20T21:36:49.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:49.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:49.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 -- 172.21.15.7:0/2823276815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff81005f0 msgr2=0x7f7ff81009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/2823276815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff81005f0 0x7f7ff81009d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7fe801acb0 tx=0x7f7fe803d150 comp rx=0 tx=0).stop 2024-04-20T21:36:49.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 -- 172.21.15.7:0/2823276815 shutdown_connections 2024-04-20T21:36:49.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/2823276815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff8100fa0 0x7f7ff8105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/2823276815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff8106270 0x7f7ff8106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/2823276815 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff81005f0 0x7f7ff81009d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.343+0000 7f7ffdc00640 1 -- 172.21.15.7:0/2823276815 >> 172.21.15.7:0/2823276815 conn(0x7f7ff8075d00 msgr2=0x7f7ff8076110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.346+0000 7f7ffdc00640 1 -- 172.21.15.7:0/2823276815 shutdown_connections 2024-04-20T21:36:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.346+0000 7f7ffdc00640 1 -- 172.21.15.7:0/2823276815 wait complete. 2024-04-20T21:36:49.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.348+0000 7f7ffdc00640 1 Processor -- start 2024-04-20T21:36:49.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.348+0000 7f7ffdc00640 1 -- start start 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ffdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 0x7f7ff8072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ffdc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 0x7f7ff8072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52042/0 (socket says 172.21.15.7:52042) 2024-04-20T21:36:49.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 0x7f7ff8072a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38896/0 (socket says 172.21.15.7:38896) 2024-04-20T21:36:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.349+0000 7f7ffdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff81a3750 0x7f7ff81a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.350+0000 7f7ff6000640 1 -- 172.21.15.7:0/760546507 learned_addr learned my addr 172.21.15.7:0/760546507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.350+0000 7f7ff7400640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff81a3750 0x7f7ff81a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.350+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff810a0c0 con 0x7f7ff81a3750 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.350+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff8109f40 con 0x7f7ff81005f0 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.350+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff810a240 con 0x7f7ff8106270 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 -- 172.21.15.7:0/760546507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 msgr2=0x7f7ff8072a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 0x7f7ff8072a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 -- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff81a3750 msgr2=0x7f7ff81a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff81a3750 0x7f7ff81a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff81a4240 con 0x7f7ff8106270 2024-04-20T21:36:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.351+0000 7f7ff6000640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e9f0 tx=0x7f7fe400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.352+0000 7f7fef400640 1 -- 172.21.15.7:0/760546507 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400ccc0 con 0x7f7ff8106270 2024-04-20T21:36:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.352+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff81a4530 con 0x7f7ff8106270 2024-04-20T21:36:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.352+0000 7f7fef400640 1 -- 172.21.15.7:0/760546507 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fe4010910 con 0x7f7ff8106270 2024-04-20T21:36:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.352+0000 7f7fef400640 1 -- 172.21.15.7:0/760546507 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4018920 con 0x7f7ff8106270 2024-04-20T21:36:49.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.352+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff81abd40 con 0x7f7ff8106270 2024-04-20T21:36:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.358+0000 7f7fef400640 1 -- 172.21.15.7:0/760546507 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7fe4010ac0 con 0x7f7ff8106270 2024-04-20T21:36:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.362+0000 7f8000a15740 1 Processor -- start 2024-04-20T21:36:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.363+0000 7f7fef400640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 0x7f7fdc0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.363+0000 7f7fef400640 1 -- 172.21.15.7:0/760546507 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7fe40d0a60 con 0x7f7ff8106270 2024-04-20T21:36:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.363+0000 7f7ff7400640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 0x7f7fdc0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.364+0000 7f7ff7400640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 0x7f7fdc0a62e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005fd0 tx=0x7f7ff0005cc0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.365+0000 7f8000a15740 1 -- start start 2024-04-20T21:36:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 0x55eb692bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb69219150 con 0x55eb692b3ac0 2024-04-20T21:36:49.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb691bf370 con 0x55eb692b31a0 2024-04-20T21:36:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f8000a15740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55eb69215110 con 0x55eb692bc6a0 2024-04-20T21:36:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.366+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38902/0 (socket says 172.21.15.7:38902) 2024-04-20T21:36:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff6a00640 1 -- 172.21.15.7:0/2859414698 learned_addr learned my addr 172.21.15.7:0/2859414698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 0x55eb692bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 -- 172.21.15.7:0/2859414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 msgr2=0x55eb692b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 -- 172.21.15.7:0/2859414698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 msgr2=0x55eb692bc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692bc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.367+0000 7f7ff7400640 1 -- 172.21.15.7:0/2859414698 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb68ff57d0 con 0x55eb692bc6a0 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.368+0000 7f7ff7400640 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 0x55eb692bea90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000bfd0 tx=0x7f7ff0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c64a0dfd369d196 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.369+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2859414698 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011b90 con 0x55eb692bc6a0 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.369+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2859414698 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0011d30 con 0x55eb692bc6a0 2024-04-20T21:36:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.369+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2859414698 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00267c0 con 0x55eb692bc6a0 2024-04-20T21:36:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.369+0000 7f8000a15740 1 -- 172.21.15.7:0/2859414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 msgr2=0x55eb692bea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.369+0000 7f8000a15740 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 0x55eb692bea90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000bfd0 tx=0x7f7ff0013000 comp rx=0 tx=0).stop 2024-04-20T21:36:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.370+0000 7f8000a15740 1 -- 172.21.15.7:0/2859414698 shutdown_connections 2024-04-20T21:36:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.370+0000 7f8000a15740 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692bc6a0 0x55eb692bea90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.370+0000 7f8000a15740 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692bc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.370+0000 7f8000a15740 1 --2- 172.21.15.7:0/2859414698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.370+0000 7f8000a15740 1 -- 172.21.15.7:0/2859414698 >> 172.21.15.7:0/2859414698 conn(0x55eb692ad8d0 msgr2=0x55eb692adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.372+0000 7f8000a15740 1 -- 172.21.15.7:0/2859414698 shutdown_connections 2024-04-20T21:36:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.372+0000 7f8000a15740 1 -- 172.21.15.7:0/2859414698 wait complete. 2024-04-20T21:36:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.373+0000 7f8000a15740 1 -- start start 2024-04-20T21:36:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.373+0000 7f8000a15740 1 Processor -- start 2024-04-20T21:36:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.374+0000 7f8000a15740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b31a0 0x55eb692b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.374+0000 7f8000a15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.375+0000 7f8000a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 0x55eb692ba1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.375+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.375+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34102/0 (socket says 172.21.15.7:34102) 2024-04-20T21:36:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.376+0000 7f7ff6000640 1 -- 172.21.15.7:0/437394498 learned_addr learned my addr 172.21.15.7:0/437394498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.376+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eb692bf490 con 0x55eb692b3ac0 2024-04-20T21:36:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.376+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb691d2740 con 0x55eb692bc6a0 2024-04-20T21:36:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.376+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55eb692bf610 con 0x55eb692b31a0 2024-04-20T21:36:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.376+0000 7f8000a15740 10 client.0 _ll_register_callbacks cb 0x55eb691cdb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.375+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b31a0 0x55eb692b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.375+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 0x55eb692ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.377+0000 7f7ff6000640 1 -- 172.21.15.7:0/437394498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b31a0 msgr2=0x55eb692b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.378+0000 7f7ff6000640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b31a0 0x55eb692b9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.378+0000 7f7ff6000640 1 -- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 msgr2=0x55eb692ba1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.378+0000 7f7ff6000640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 0x55eb692ba1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.378+0000 7f7ff6000640 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb68f2b3f0 con 0x55eb692b3ac0 2024-04-20T21:36:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.379+0000 7f7ff7400640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 0x55eb692ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:49.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.379+0000 7f7ff6000640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4098f00 tx=0x7f7fe409c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e304d02715eead7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.379+0000 7f7fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:36:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.379+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe40ce620 con 0x55eb692b3ac0 2024-04-20T21:36:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.380+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff0011d30 con 0x55eb692b3ac0 2024-04-20T21:36:49.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.380+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7fe40bbbb0 con 0x55eb692b3ac0 2024-04-20T21:36:49.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.381+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7fe40b7020 con 0x55eb692b3ac0 2024-04-20T21:36:49.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.381+0000 7f8000a15740 10 client.15852 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:49.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.381+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb68eeb0f0 con 0x55eb692b3ac0 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.382+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0011d30 con 0x55eb692b3ac0 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.382+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f7fe40bb070 con 0x55eb692b3ac0 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.382+0000 7f8000a15740 10 client.15852 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.382+0000 7f8000a15740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.382+0000 7f8000a15740 20 client.15852 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.383+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692cca20 0x55eb692cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.383+0000 7f8000a15740 4 client.15852 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.383+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb69226b70 con 0x55eb692cca20 2024-04-20T21:36:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.383+0000 7f7ff6a00640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692cca20 0x55eb692cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.384+0000 7f7ff6a00640 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692cca20 0x55eb692cee50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f3eaa3b795ddfd server_cookie=a9bfaaa925be8b29 in_seq=0 out_seq=0 2024-04-20T21:36:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.384+0000 7f7fe0c00640 10 client.15852 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.384+0000 7f7fe0c00640 1 -- 172.21.15.7:0/437394498 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb69226b70 con 0x55eb692cca20 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.384+0000 7f7fe0c00640 10 client.15852 handle_command_reply: tid=0 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 client.15852 shutdown 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 msgr2=0x55eb692b9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4098f00 tx=0x7f7fe409c6f0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 shutdown_connections 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692bc6a0 0x55eb692ba1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692cca20 0x55eb692cee50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b3ac0 0x55eb692b9c70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 --2- 172.21.15.7:0/437394498 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b31a0 0x55eb692b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 >> 172.21.15.7:0/437394498 conn(0x55eb692ad8d0 msgr2=0x55eb692af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f7fe0c00640 0 client.15852 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:49.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 shutdown_connections 2024-04-20T21:36:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 1 -- 172.21.15.7:0/437394498 wait complete. 2024-04-20T21:36:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.385+0000 7f8000a15740 20 client.15852 trim_cache size 0 max 16384 2024-04-20T21:36:49.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 Processor -- start 2024-04-20T21:36:49.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 -- start start 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 0x55eb692c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b6c20 con 0x55eb692b31a0 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b44e0 con 0x55eb692b3ac0 2024-04-20T21:36:49.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.415+0000 7f8000a15740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b2330 con 0x55eb692ad8d0 2024-04-20T21:36:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38928/0 (socket says 172.21.15.7:38928) 2024-04-20T21:36:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52070/0 (socket says 172.21.15.7:52070) 2024-04-20T21:36:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 -- 172.21.15.7:0/1932737976 learned_addr learned my addr 172.21.15.7:0/1932737976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff6000640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 0x55eb692c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 -- 172.21.15.7:0/1932737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 msgr2=0x55eb692bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 -- 172.21.15.7:0/1932737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 msgr2=0x55eb692c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 0x55eb692c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff7400640 1 -- 172.21.15.7:0/1932737976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb692b2b70 con 0x55eb692b3ac0 2024-04-20T21:36:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.416+0000 7f7ff6a00640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.417+0000 7f7ff6000640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 0x55eb692c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.417+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0003b00 tx=0x7f7ff0011990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19a5d61f46b39c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.417+0000 7f7fe3400640 1 -- 172.21.15.7:0/1932737976 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011d80 con 0x55eb692b3ac0 2024-04-20T21:36:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.417+0000 7f7fe3400640 1 -- 172.21.15.7:0/1932737976 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0005520 con 0x55eb692b3ac0 2024-04-20T21:36:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.417+0000 7f7fe3400640 1 -- 172.21.15.7:0/1932737976 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff001e420 con 0x55eb692b3ac0 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 -- 172.21.15.7:0/1932737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 msgr2=0x55eb692adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0003b00 tx=0x7f7ff0011990 comp rx=0 tx=0).stop 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 -- 172.21.15.7:0/1932737976 shutdown_connections 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b3ac0 0x55eb692adcb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692b31a0 0x55eb692c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 --2- 172.21.15.7:0/1932737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692ad8d0 0x55eb692bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.418+0000 7f8000a15740 1 -- 172.21.15.7:0/1932737976 >> 172.21.15.7:0/1932737976 conn(0x55eb692b5100 msgr2=0x55eb692aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.420+0000 7f8000a15740 1 -- 172.21.15.7:0/1932737976 shutdown_connections 2024-04-20T21:36:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.420+0000 7f8000a15740 1 -- 172.21.15.7:0/1932737976 wait complete. 2024-04-20T21:36:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.420+0000 7f8000a15740 1 -- start start 2024-04-20T21:36:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.421+0000 7f8000a15740 1 Processor -- start 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.421+0000 7f8000a15740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692ad8d0 0x55eb692c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.421+0000 7f8000a15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.421+0000 7f8000a15740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.422+0000 7f8000a15740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b29a0 con 0x55eb692ad8d0 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.422+0000 7f8000a15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b3e10 con 0x55eb692b31a0 2024-04-20T21:36:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.422+0000 7f8000a15740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff81b3070 con 0x55eb692b3ac0 2024-04-20T21:36:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.422+0000 7f8000a15740 10 client.0 _ll_register_callbacks cb 0x7f7fe4098d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:49.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.423+0000 7f7ff6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692ad8d0 0x55eb692c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52082/0 (socket says 172.21.15.7:52082) 2024-04-20T21:36:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 -- 172.21.15.7:0/1160552704 learned_addr learned my addr 172.21.15.7:0/1160552704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 -- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 msgr2=0x55eb692c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 -- 172.21.15.7:0/1160552704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692ad8d0 msgr2=0x55eb692c0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692ad8d0 0x55eb692c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.424+0000 7f7ff7400640 1 -- 172.21.15.7:0/1160552704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb69331fd0 con 0x55eb692b3ac0 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f7ff6000640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f7ff7400640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0021040 tx=0x7f7ff00048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ed63a2669e0d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f7fe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00051a0 con 0x55eb692b3ac0 2024-04-20T21:36:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7ff001e420 con 0x55eb692b3ac0 2024-04-20T21:36:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.425+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff0005520 con 0x55eb692b3ac0 2024-04-20T21:36:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.426+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ff00041a0 con 0x55eb692b3ac0 2024-04-20T21:36:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.426+0000 7f8000a15740 10 client.25975 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.426+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb693331f0 con 0x55eb692b3ac0 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.426+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff0005520 con 0x55eb692b3ac0 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f7ff00315b0 con 0x55eb692b3ac0 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 10 client.25975 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 20 client.25975 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692bc6a0 0x55eb693356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 4 client.25975 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.427+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb692209e0 con 0x55eb692bc6a0 2024-04-20T21:36:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.428+0000 7f7ff6a00640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692bc6a0 0x55eb693356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.429+0000 7f7ff6a00640 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692bc6a0 0x55eb693356e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1808a6fcdff41670 server_cookie=4c123de452f3262e in_seq=0 out_seq=0 2024-04-20T21:36:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.429+0000 7f7fe0c00640 10 client.25975 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.429+0000 7f7fe0c00640 1 -- 172.21.15.7:0/1160552704 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55eb692209e0 con 0x55eb692bc6a0 2024-04-20T21:36:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.429+0000 7f7fe0c00640 10 client.25975 handle_command_reply: tid=0 2024-04-20T21:36:49.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.429+0000 7f8000a15740 1 client.25975 shutdown 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.431+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 msgr2=0x55eb692c0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.431+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0021040 tx=0x7f7ff00048d0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.431+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 shutdown_connections 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.431+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb692b31a0 0x55eb692c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.431+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55eb692bc6a0 0x55eb693356e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55eb692b3ac0 0x55eb692c0ca0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f8000a15740 1 --2- 172.21.15.7:0/1160552704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eb692ad8d0 0x55eb692c0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f7fe0c00640 0 client.25975 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 >> 172.21.15.7:0/1160552704 conn(0x55eb692b5100 msgr2=0x55eb692af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 shutdown_connections 2024-04-20T21:36:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.432+0000 7f8000a15740 1 -- 172.21.15.7:0/1160552704 wait complete. 2024-04-20T21:36:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.433+0000 7f8000a15740 20 client.25975 trim_cache size 0 max 16384 2024-04-20T21:36:49.432 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:36:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:36:49.432 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 msgr2=0x7f7fdc0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 0x7f7fdc0a62e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0005fd0 tx=0x7f7ff0005cc0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 msgr2=0x7f7ff8072fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e9f0 tx=0x7f7fe400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 shutdown_connections 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ff81a3750 0x7f7ff81a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7ff8106270 0x7f7ff8072fd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7fdc0a3e20 0x7f7fdc0a62e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 --2- 172.21.15.7:0/760546507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ff81005f0 0x7f7ff8072a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 >> 172.21.15.7:0/760546507 conn(0x7f7ff8075d00 msgr2=0x7f7ff8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.435+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 shutdown_connections 2024-04-20T21:36:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.436+0000 7f7ffdc00640 1 -- 172.21.15.7:0/760546507 wait complete. 2024-04-20T21:36:49.463 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:36:49.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 -- 172.21.15.7:0/3066421344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc11b740 msgr2=0x7f6ffc11db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 --2- 172.21.15.7:0/3066421344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc11b740 0x7f6ffc11db30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009be0 tx=0x7f6ff802bf10 comp rx=0 tx=0).stop 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 -- 172.21.15.7:0/3066421344 shutdown_connections 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 --2- 172.21.15.7:0/3066421344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc11b740 0x7f6ffc11db30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 --2- 172.21.15.7:0/3066421344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.777+0000 7f7004800640 1 --2- 172.21.15.7:0/3066421344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc110780 0x7f6ffc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.778+0000 7f7004800640 1 -- 172.21.15.7:0/3066421344 >> 172.21.15.7:0/3066421344 conn(0x7f6ffc06e2f0 msgr2=0x7f6ffc06e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.778+0000 7f7004800640 1 -- 172.21.15.7:0/3066421344 shutdown_connections 2024-04-20T21:36:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.778+0000 7f7004800640 1 -- 172.21.15.7:0/3066421344 wait complete. 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.780+0000 7f7004800640 1 Processor -- start 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.780+0000 7f7004800640 1 -- start start 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7004800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc118c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7004800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7004800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 0x7f6ffc111ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7001400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7001400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52096/0 (socket says 172.21.15.7:52096) 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.781+0000 7f7002800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 0x7f6ffc111ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc118c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 -- 172.21.15.7:0/755980511 learned_addr learned my addr 172.21.15.7:0/755980511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1211d0 con 0x7f6ffc11b740 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc121050 con 0x7f6ffc1094a0 2024-04-20T21:36:49.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc121350 con 0x7f6ffc110780 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 -- 172.21.15.7:0/755980511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 msgr2=0x7f6ffc118c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc118c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 -- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 msgr2=0x7f6ffc111ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 0x7f6ffc111ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7001400640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc112760 con 0x7f6ffc110780 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.782+0000 7f7002800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 0x7f6ffc111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:49.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.783+0000 7f7001400640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000e9f0 tx=0x7f6ff000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.783+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000ccc0 con 0x7f6ffc110780 2024-04-20T21:36:49.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.784+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc112a50 con 0x7f6ffc110780 2024-04-20T21:36:49.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.784+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc116a70 con 0x7f6ffc110780 2024-04-20T21:36:49.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.784+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0010910 con 0x7f6ffc110780 2024-04-20T21:36:49.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.784+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00188a0 con 0x7f6ffc110780 2024-04-20T21:36:49.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.787+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe0005290 con 0x7f6ffc110780 2024-04-20T21:36:49.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.792+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6ff0010430 con 0x7f6ffc110780 2024-04-20T21:36:49.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.794+0000 7f6ff6a00640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 0x7f6fe40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:49.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.794+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6ff00d0b30 con 0x7f6ffc110780 2024-04-20T21:36:49.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.795+0000 7f7001e00640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 0x7f6fe40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.798+0000 7f7001e00640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 0x7f6fe40a6320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005fd0 tx=0x7f6fec009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:49.800+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f6ff009d1c0 con 0x7f6ffc110780 2024-04-20T21:36:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.384+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fe0005850 con 0x7f6ffc110780 2024-04-20T21:36:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.387+0000 7f6ff6a00640 1 -- 172.21.15.7:0/755980511 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11872 (secure 0 0 0) 0x7f6ff009cac0 con 0x7f6ffc110780 2024-04-20T21:36:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:36:50.386 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:35:06.444983+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24559,24575,24535]},"id":1}]} 2024-04-20T21:36:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2024-04-20T21:36:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.391+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 msgr2=0x7f6fe40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 0x7f6fe40a6320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005fd0 tx=0x7f6fec009260 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 msgr2=0x7f6ffc1191a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000e9f0 tx=0x7f6ff000edf0 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 shutdown_connections 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ffc11b740 0x7f6ffc111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ffc110780 0x7f6ffc1191a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6fe40a3e60 0x7f6fe40a6320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 --2- 172.21.15.7:0/755980511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc1094a0 0x7f6ffc118c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.392+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 >> 172.21.15.7:0/755980511 conn(0x7f6ffc06e2f0 msgr2=0x7f6ffc10e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.394+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 shutdown_connections 2024-04-20T21:36:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:50.394+0000 7f7004800640 1 -- 172.21.15.7:0/755980511 wait complete. 2024-04-20T21:36:50.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:36:50.602 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:50 smithi007 ceph-mon[16867]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 157 MiB/s wr, 23.08k op/s 2024-04-20T21:36:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:50 smithi016 ceph-mon[21409]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 157 MiB/s wr, 23.08k op/s 2024-04-20T21:36:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:50 smithi186 ceph-mon[22157]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 157 MiB/s wr, 23.08k op/s 2024-04-20T21:36:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.011+0000 7fba2fe00640 1 -- 172.21.15.7:0/1063473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba301094a0 msgr2=0x7fba30109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.011+0000 7fba2fe00640 1 --2- 172.21.15.7:0/1063473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba301094a0 0x7fba30109920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.013+0000 7fba2fe00640 1 -- 172.21.15.7:0/1063473077 shutdown_connections 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.013+0000 7fba2fe00640 1 --2- 172.21.15.7:0/1063473077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba301094a0 0x7fba30109920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.013+0000 7fba2fe00640 1 --2- 172.21.15.7:0/1063473077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba30116f60 0x7fba30119350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.013+0000 7fba2fe00640 1 --2- 172.21.15.7:0/1063473077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba3010c4a0 0x7fba30108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.013+0000 7fba2fe00640 1 -- 172.21.15.7:0/1063473077 >> 172.21.15.7:0/1063473077 conn(0x7fba3006ce70 msgr2=0x7fba3006d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.014+0000 7fba2fe00640 1 -- 172.21.15.7:0/1063473077 shutdown_connections 2024-04-20T21:36:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.014+0000 7fba2fe00640 1 -- 172.21.15.7:0/1063473077 wait complete. 2024-04-20T21:36:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.025+0000 7fba2fe00640 1 Processor -- start 2024-04-20T21:36:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.025+0000 7fba2fe00640 1 -- start start 2024-04-20T21:36:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.025+0000 7fba2fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba301094a0 0x7fba30116770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.026+0000 7fba2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.028+0000 7fba2fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30116f60 0x7fba30113d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.028+0000 7fba2fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba3011cb90 con 0x7fba301094a0 2024-04-20T21:36:51.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.028+0000 7fba2fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba3011ca10 con 0x7fba3010c4a0 2024-04-20T21:36:51.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.028+0000 7fba2fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fba3011cd10 con 0x7fba30116f60 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.029+0000 7fba2e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.029+0000 7fba2e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38960/0 (socket says 172.21.15.7:38960) 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.029+0000 7fba2e000640 1 -- 172.21.15.7:0/460209555 learned_addr learned my addr 172.21.15.7:0/460209555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2f400640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30116f60 0x7fba30113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2e000640 1 -- 172.21.15.7:0/460209555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30116f60 msgr2=0x7fba30113d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2e000640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30116f60 0x7fba30113d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2e000640 1 -- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba301094a0 msgr2=0x7fba30116770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2e000640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba301094a0 0x7fba30116770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.030+0000 7fba2e000640 1 -- 172.21.15.7:0/460209555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba30114610 con 0x7fba3010c4a0 2024-04-20T21:36:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.033+0000 7fba2e000640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba1c004f50 tx=0x7fba1c004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.039+0000 7fba34a00640 1 -- 172.21.15.7:0/460209555 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c005210 con 0x7fba3010c4a0 2024-04-20T21:36:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.039+0000 7fba34a00640 1 -- 172.21.15.7:0/460209555 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c044070 con 0x7fba3010c4a0 2024-04-20T21:36:51.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.039+0000 7fba34a00640 1 -- 172.21.15.7:0/460209555 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0347a0 con 0x7fba3010c4a0 2024-04-20T21:36:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.041+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba301b3890 con 0x7fba3010c4a0 2024-04-20T21:36:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.041+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba301b3d60 con 0x7fba3010c4a0 2024-04-20T21:36:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.066+0000 7fba38b69740 1 Processor -- start 2024-04-20T21:36:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.067+0000 7fba38b69740 1 -- start start 2024-04-20T21:36:51.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.067+0000 7fba38b69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.070+0000 7fba34a00640 1 -- 172.21.15.7:0/460209555 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fba1c0053b0 con 0x7fba3010c4a0 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba38b69740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba38b69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba38b69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562403378050 con 0x56240342d680 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba38b69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56240331e490 con 0x562403412e20 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba38b69740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562403374010 con 0x562403413740 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38974/0 (socket says 172.21.15.7:38974) 2024-04-20T21:36:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.073+0000 7fba2f400640 1 -- 172.21.15.7:0/1246281204 learned_addr learned my addr 172.21.15.7:0/1246281204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.074+0000 7fba2ea00640 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.075+0000 7fba2f400640 1 -- 172.21.15.7:0/1246281204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 msgr2=0x56240342d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.075+0000 7fba2f400640 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.075+0000 7fba2f400640 1 -- 172.21.15.7:0/1246281204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 msgr2=0x56240342fa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.075+0000 7fba2f400640 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342fa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.075+0000 7fba2f400640 1 -- 172.21.15.7:0/1246281204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562403326070 con 0x562403412e20 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.081+0000 7fba2f400640 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba2000ca90 tx=0x7fba2000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c686ab464c9f6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.081+0000 7fba1be00640 1 -- 172.21.15.7:0/1246281204 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20004020 con 0x562403412e20 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.082+0000 7fba1be00640 1 -- 172.21.15.7:0/1246281204 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba200041c0 con 0x562403412e20 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.082+0000 7fba34a00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 0x7fba140a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.083+0000 7fba38b69740 1 -- 172.21.15.7:0/1246281204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 msgr2=0x562403413200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.083+0000 7fba38b69740 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba2000ca90 tx=0x7fba2000cf50 comp rx=0 tx=0).stop 2024-04-20T21:36:51.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.084+0000 7fba34a00640 1 -- 172.21.15.7:0/460209555 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fba1c0f2130 con 0x7fba3010c4a0 2024-04-20T21:36:51.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.086+0000 7fba2ea00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 0x7fba140a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.089+0000 7fba2ea00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 0x7fba140a63b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba28005fd0 tx=0x7fba280073c0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.086+0000 7fba38b69740 1 -- 172.21.15.7:0/1246281204 shutdown_connections 2024-04-20T21:36:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.090+0000 7fba38b69740 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.090+0000 7fba38b69740 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.090+0000 7fba38b69740 1 --2- 172.21.15.7:0/1246281204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x562403413200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.090+0000 7fba38b69740 1 -- 172.21.15.7:0/1246281204 >> 172.21.15.7:0/1246281204 conn(0x56240340d8a0 msgr2=0x56240340dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.091+0000 7fba38b69740 1 -- 172.21.15.7:0/1246281204 shutdown_connections 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.091+0000 7fba38b69740 1 -- 172.21.15.7:0/1246281204 wait complete. 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.092+0000 7fba38b69740 1 -- start start 2024-04-20T21:36:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.092+0000 7fba38b69740 1 Processor -- start 2024-04-20T21:36:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.098+0000 7fba38b69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x56240342a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.100+0000 7fba38b69740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.101+0000 7fba2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x56240342a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.101+0000 7fba2ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x56240342a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38978/0 (socket says 172.21.15.7:38978) 2024-04-20T21:36:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.101+0000 7fba2ea00640 1 -- 172.21.15.7:0/869294260 learned_addr learned my addr 172.21.15.7:0/869294260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.101+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.101+0000 7fba2e000640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.104+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562403430510 con 0x56240342d680 2024-04-20T21:36:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.104+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562403331860 con 0x562403412e20 2024-04-20T21:36:51.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.104+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x562403430690 con 0x562403413740 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.104+0000 7fba38b69740 10 client.0 _ll_register_callbacks cb 0x5624032fffe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 -- 172.21.15.7:0/869294260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 msgr2=0x56240342a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x56240342a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 -- 172.21.15.7:0/869294260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 msgr2=0x56240342b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562403335950 con 0x562403413740 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba18c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.105+0000 7fba2e000640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342ab60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0ef980 tx=0x7fba1c0bdd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5badb1e46270bfb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.106+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0efcf0 con 0x562403413740 2024-04-20T21:36:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.107+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fba1c0efaf0 con 0x562403413740 2024-04-20T21:36:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.108+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba200041c0 con 0x562403413740 2024-04-20T21:36:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.108+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba1c0be8d0 con 0x562403413740 2024-04-20T21:36:51.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.110+0000 7fba38b69740 10 client.25990 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:51.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.110+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562403303820 con 0x562403413740 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.113+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba200041c0 con 0x562403413740 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.114+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fba1c0c69b0 con 0x562403413740 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 10 client.25990 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 10 client.25990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 20 client.25990 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56240343d970 0x56240343fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 4 client.25990 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:51.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.115+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562403385a70 con 0x56240343d970 2024-04-20T21:36:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.118+0000 7fba2ea00640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56240343d970 0x56240343fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.120+0000 7fba2ea00640 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56240343d970 0x56240343fd80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5b12ad82e25f3a5 server_cookie=32bfa53f3cf5fa36 in_seq=0 out_seq=0 2024-04-20T21:36:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba18c00640 10 client.25990 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba18c00640 1 -- 172.21.15.7:0/869294260 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562403385a70 con 0x56240343d970 2024-04-20T21:36:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba18c00640 10 client.25990 handle_command_reply: tid=0 2024-04-20T21:36:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba38b69740 1 client.25990 shutdown 2024-04-20T21:36:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 msgr2=0x56240342ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.123+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342ab60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0ef980 tx=0x7fba1c0bdd10 comp rx=0 tx=0).stop 2024-04-20T21:36:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 shutdown_connections 2024-04-20T21:36:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x56240342b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403413740 0x56240342ab60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56240343d970 0x56240343fd80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba18c00640 0 client.25990 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 --2- 172.21.15.7:0/869294260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403412e20 0x56240342a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 >> 172.21.15.7:0/869294260 conn(0x56240340d8a0 msgr2=0x56240340f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 shutdown_connections 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 1 -- 172.21.15.7:0/869294260 wait complete. 2024-04-20T21:36:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.124+0000 7fba38b69740 20 client.25990 trim_cache size 0 max 16384 2024-04-20T21:36:51.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.173+0000 7fba38b69740 1 Processor -- start 2024-04-20T21:36:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.173+0000 7fba38b69740 1 -- start start 2024-04-20T21:36:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.174+0000 7fba38b69740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x56240342cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.174+0000 7fba38b69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403413740 0x56240342aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.174+0000 7fba38b69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.174+0000 7fba38b69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba301b5dc0 con 0x56240342d680 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.175+0000 7fba38b69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba301b6490 con 0x562403413740 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.175+0000 7fba38b69740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fba301aa590 con 0x562403412e20 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.175+0000 7fba2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.175+0000 7fba2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34170/0 (socket says 172.21.15.7:34170) 2024-04-20T21:36:51.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.175+0000 7fba2f400640 1 -- 172.21.15.7:0/1784813347 learned_addr learned my addr 172.21.15.7:0/1784813347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 -- 172.21.15.7:0/1784813347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 msgr2=0x56240342cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x56240342cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 -- 172.21.15.7:0/1784813347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403413740 msgr2=0x56240342aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403413740 0x56240342aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 -- 172.21.15.7:0/1784813347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240308a3f0 con 0x56240342d680 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.176+0000 7fba2f400640 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c790 tx=0x7fba2000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ed0046e9a77426 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.177+0000 7fba1b400640 1 -- 172.21.15.7:0/1784813347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000ed20 con 0x56240342d680 2024-04-20T21:36:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.177+0000 7fba1b400640 1 -- 172.21.15.7:0/1784813347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20010d10 con 0x56240342d680 2024-04-20T21:36:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.177+0000 7fba1b400640 1 -- 172.21.15.7:0/1784813347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2001ad10 con 0x56240342d680 2024-04-20T21:36:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 -- 172.21.15.7:0/1784813347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 msgr2=0x562403445690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c790 tx=0x7fba2000c7c0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 -- 172.21.15.7:0/1784813347 shutdown_connections 2024-04-20T21:36:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403413740 0x56240342aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403445690 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 --2- 172.21.15.7:0/1784813347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x56240342cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.178+0000 7fba38b69740 1 -- 172.21.15.7:0/1784813347 >> 172.21.15.7:0/1784813347 conn(0x56240340d8a0 msgr2=0x56240340c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.184+0000 7fba38b69740 1 -- 172.21.15.7:0/1784813347 shutdown_connections 2024-04-20T21:36:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.184+0000 7fba38b69740 1 -- 172.21.15.7:0/1784813347 wait complete. 2024-04-20T21:36:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.185+0000 7fba38b69740 1 -- start start 2024-04-20T21:36:51.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.185+0000 7fba38b69740 1 Processor -- start 2024-04-20T21:36:51.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x5624034313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403431930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fba301a8ad0 con 0x56240342d680 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba301a9ec0 con 0x562403431e70 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fba301a9120 con 0x562403412e20 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba38b69740 10 client.0 _ll_register_callbacks cb 0x7fba1c0bdfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba2e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403431930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba2f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39004/0 (socket says 172.21.15.7:39004) 2024-04-20T21:36:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba2f400640 1 -- 172.21.15.7:0/3830790454 learned_addr learned my addr 172.21.15.7:0/3830790454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.187+0000 7fba2ea00640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x5624034313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.188+0000 7fba2f400640 1 -- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 msgr2=0x5624034313f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.188+0000 7fba2f400640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x5624034313f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.188+0000 7fba2f400640 1 -- 172.21.15.7:0/3830790454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 msgr2=0x562403431930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.188+0000 7fba2f400640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403431930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.188+0000 7fba2f400640 1 -- 172.21.15.7:0/3830790454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624031547d0 con 0x562403431e70 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.189+0000 7fba2e000640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403431930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.189+0000 7fba2f400640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba2001a640 tx=0x7fba2001a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f870e4f39842e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.189+0000 7fba18c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.189+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20010a40 con 0x562403431e70 2024-04-20T21:36:51.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.190+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fba20005050 con 0x562403431e70 2024-04-20T21:36:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.192+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba20010d10 con 0x562403431e70 2024-04-20T21:36:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.192+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fba20005250 con 0x562403431e70 2024-04-20T21:36:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.192+0000 7fba38b69740 10 client.26015 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.192+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624034a4450 con 0x562403431e70 2024-04-20T21:36:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20010d10 con 0x562403431e70 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fba20004860 con 0x562403431e70 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba38b69740 10 client.26015 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba38b69740 10 client.26015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba38b69740 20 client.26015 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5624034a4a50 0x5624034a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.193+0000 7fba38b69740 4 client.26015 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:51.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.194+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56240337f8e0 con 0x5624034a4a50 2024-04-20T21:36:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.194+0000 7fba2ea00640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5624034a4a50 0x5624034a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.195+0000 7fba2ea00640 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5624034a4a50 0x5624034a6e70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57dcbd7d5ac75f80 server_cookie=beee781d3868a211 in_seq=0 out_seq=0 2024-04-20T21:36:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.195+0000 7fba18c00640 10 client.26015 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.195+0000 7fba18c00640 1 -- 172.21.15.7:0/3830790454 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56240337f8e0 con 0x5624034a4a50 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.195+0000 7fba18c00640 10 client.26015 handle_command_reply: tid=0 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.195+0000 7fba38b69740 1 client.26015 shutdown 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 msgr2=0x5624034a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fba2001a640 tx=0x7fba2001a670 comp rx=0 tx=0).stop 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 shutdown_connections 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5624034a4a50 0x5624034a6e70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562403431e70 0x5624034a2f00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba18c00640 0 client.26015 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56240342d680 0x562403431930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 --2- 172.21.15.7:0/3830790454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x562403412e20 0x5624034313f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 >> 172.21.15.7:0/3830790454 conn(0x56240340d8a0 msgr2=0x56240340c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 shutdown_connections 2024-04-20T21:36:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 1 -- 172.21.15.7:0/3830790454 wait complete. 2024-04-20T21:36:51.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.196+0000 7fba38b69740 20 client.26015 trim_cache size 0 max 16384 2024-04-20T21:36:51.196 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.197+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 msgr2=0x7fba140a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.197+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 0x7fba140a63b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba28005fd0 tx=0x7fba280073c0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.197+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 msgr2=0x7fba30113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.197+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba1c004f50 tx=0x7fba1c004430 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 shutdown_connections 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fba140a3ef0 0x7fba140a63b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba3010c4a0 0x7fba30113810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba30116f60 0x7fba30113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 --2- 172.21.15.7:0/460209555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba301094a0 0x7fba30116770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 >> 172.21.15.7:0/460209555 conn(0x7fba3006ce70 msgr2=0x7fba30118430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 shutdown_connections 2024-04-20T21:36:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.198+0000 7fba2fe00640 1 -- 172.21.15.7:0/460209555 wait complete. 2024-04-20T21:36:51.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:36:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 -- 172.21.15.7:0/1764943665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 msgr2=0x7f1fb0115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 --2- 172.21.15.7:0/1764943665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb0115210 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009c10 tx=0x7f1fa802bef0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 -- 172.21.15.7:0/1764943665 shutdown_connections 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 --2- 172.21.15.7:0/1764943665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb0115210 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 --2- 172.21.15.7:0/1764943665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb01084a0 0x7f1fb0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 --2- 172.21.15.7:0/1764943665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb010e3e0 0x7f1fb010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 -- 172.21.15.7:0/1764943665 >> 172.21.15.7:0/1764943665 conn(0x7f1fb0077d70 msgr2=0x7f1fb0106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.420+0000 7f1fafe00640 1 -- 172.21.15.7:0/1764943665 shutdown_connections 2024-04-20T21:36:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.421+0000 7f1fafe00640 1 -- 172.21.15.7:0/1764943665 wait complete. 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.421+0000 7f1fafe00640 1 Processor -- start 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 -- start start 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01084a0 0x7f1fb01a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb010e3e0 0x7f1fb019ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0118710 con 0x7f1fb0108e40 2024-04-20T21:36:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0118590 con 0x7f1fb01084a0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fafe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb0118890 con 0x7f1fb010e3e0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fae000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fae000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34190/0 (socket says 172.21.15.7:34190) 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1faf400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb010e3e0 0x7f1fb019ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1fae000640 1 -- 172.21.15.7:0/2345813575 learned_addr learned my addr 172.21.15.7:0/2345813575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.422+0000 7f1faea00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01084a0 0x7f1fb01a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 -- 172.21.15.7:0/2345813575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb010e3e0 msgr2=0x7f1fb019ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb010e3e0 0x7f1fb019ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 -- 172.21.15.7:0/2345813575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01084a0 msgr2=0x7f1fb01a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01084a0 0x7f1fb01a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 -- 172.21.15.7:0/2345813575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb019f190 con 0x7f1fb0108e40 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fae000640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e9f0 tx=0x7f1f9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fa7400640 1 -- 172.21.15.7:0/2345813575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00ccc0 con 0x7f1fb0108e40 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fa7400640 1 -- 172.21.15.7:0/2345813575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f9c005500 con 0x7f1fb0108e40 2024-04-20T21:36:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fa7400640 1 -- 172.21.15.7:0/2345813575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c010690 con 0x7f1fb0108e40 2024-04-20T21:36:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb019f480 con 0x7f1fb0108e40 2024-04-20T21:36:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.423+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb01ab720 con 0x7f1fb0108e40 2024-04-20T21:36:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.428+0000 7f1fb826e740 1 Processor -- start 2024-04-20T21:36:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.428+0000 7f1fb826e740 1 -- start start 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 0x560e9467b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e9467da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e945d7050 con 0x560e94672a90 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e9457d490 con 0x560e9467b670 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fb826e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560e945d3010 con 0x560e94672170 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1faf400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1faea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 0x560e9467b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1fa7400640 1 -- 172.21.15.7:0/2345813575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1f9c0046c0 con 0x7f1fb0108e40 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1faf400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52202/0 (socket says 172.21.15.7:52202) 2024-04-20T21:36:51.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.429+0000 7f1faf400640 1 -- 172.21.15.7:0/1069027876 learned_addr learned my addr 172.21.15.7:0/1069027876 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1fae000640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e9467da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 -- 172.21.15.7:0/1069027876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 msgr2=0x560e94672550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 -- 172.21.15.7:0/1069027876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 msgr2=0x560e9467da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e9467da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 -- 172.21.15.7:0/1069027876 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e94585070 con 0x560e94672a90 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1fae000640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e9467da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faf400640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1faea00640 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 0x560e9467b130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009ac0 tx=0x7f1fa0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961afbe5b2b5e800 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1f9b400640 1 -- 172.21.15.7:0/1069027876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0011da0 con 0x560e94672a90 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.430+0000 7f1f9b400640 1 -- 172.21.15.7:0/1069027876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa0012930 con 0x560e94672a90 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1f9b400640 1 -- 172.21.15.7:0/1069027876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa000f9c0 con 0x560e94672a90 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fa7400640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 0x7f1f940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fa7400640 1 -- 172.21.15.7:0/2345813575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1f9c0d0af0 con 0x7f1fb0108e40 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 -- 172.21.15.7:0/1069027876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 msgr2=0x560e9467b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 0x560e9467b130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009ac0 tx=0x7f1fa0009f80 comp rx=0 tx=0).stop 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1faf400640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 0x7f1f940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 -- 172.21.15.7:0/1069027876 shutdown_connections 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e9467da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672a90 0x560e9467b130 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 --2- 172.21.15.7:0/1069027876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e94672550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.431+0000 7f1fb826e740 1 -- 172.21.15.7:0/1069027876 >> 172.21.15.7:0/1069027876 conn(0x560e9466c8a0 msgr2=0x560e9466cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.432+0000 7f1fb826e740 1 -- 172.21.15.7:0/1069027876 shutdown_connections 2024-04-20T21:36:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.432+0000 7f1faf400640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 0x7f1f940a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80099a0 tx=0x7f1fa8005fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.432+0000 7f1fb826e740 1 -- 172.21.15.7:0/1069027876 wait complete. 2024-04-20T21:36:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 -- start start 2024-04-20T21:36:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 Processor -- start 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94678bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e94679100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e9467e460 con 0x560e94672170 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e94590860 con 0x560e9467b670 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560e9467e5e0 con 0x560e94672a90 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fb826e740 10 client.0 _ll_register_callbacks cb 0x560e9455efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1faea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1fae000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94678bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1faea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34216/0 (socket says 172.21.15.7:34216) 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1faf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e94679100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.433+0000 7f1faea00640 1 -- 172.21.15.7:0/2939461636 learned_addr learned my addr 172.21.15.7:0/2939461636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 -- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 msgr2=0x560e94678bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94678bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 -- 172.21.15.7:0/2939461636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 msgr2=0x560e94679100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e94679100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 -- 172.21.15.7:0/2939461636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e94594950 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1faea00640 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1fa001c040 tx=0x7f1fa0012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f09658cc7c5b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1f8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.434+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0028070 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1fa0004020 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa0012930 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fa0033050 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1fb826e740 10 client.15876 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e94562820 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa0012930 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.435+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f1fa000f800 con 0x560e94672170 2024-04-20T21:36:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:36:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 20 client.15876 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e9468b500 0x560e9468d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 4 client.15876 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e945e4a70 con 0x560e9468b500 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.436+0000 7f1fae000640 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e9468b500 0x560e9468d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.437+0000 7f1fae000640 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e9468b500 0x560e9468d8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c49d81a36efcee5 server_cookie=6120d7d261f08493 in_seq=0 out_seq=0 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.437+0000 7f1f8fe00640 10 client.15876 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.438+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2939461636 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e945e4a70 con 0x560e9468b500 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.438+0000 7f1f8fe00640 10 client.15876 handle_command_reply: tid=0 2024-04-20T21:36:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.438+0000 7f1fb826e740 1 client.15876 shutdown 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 msgr2=0x560e94678680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1fa001c040 tx=0x7f1fa0012680 comp rx=0 tx=0).stop 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 shutdown_connections 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e9468b500 0x560e9468d8f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e9467b670 0x560e94679100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94678bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2939461636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e94672170 0x560e94678680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1f8fe00640 0 client.15876 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 >> 172.21.15.7:0/2939461636 conn(0x560e9466c8a0 msgr2=0x560e9466e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 shutdown_connections 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 1 -- 172.21.15.7:0/2939461636 wait complete. 2024-04-20T21:36:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.439+0000 7f1fb826e740 20 client.15876 trim_cache size 0 max 16384 2024-04-20T21:36:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.454+0000 7f1fb826e740 1 Processor -- start 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.454+0000 7f1fb826e740 1 -- start start 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e9467adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672a90 0x560e94687320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b68b0 con 0x560e9467b670 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b4170 con 0x560e94672a90 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fb826e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b1fc0 con 0x560e94672170 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e9467adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34228/0 (socket says 172.21.15.7:34228) 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1fae000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672a90 0x560e94687320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 -- 172.21.15.7:0/4074997323 learned_addr learned my addr 172.21.15.7:0/4074997323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 -- 172.21.15.7:0/4074997323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 msgr2=0x560e9467adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e9467adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 -- 172.21.15.7:0/4074997323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672a90 msgr2=0x560e94687320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672a90 0x560e94687320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.455+0000 7f1faf400640 1 -- 172.21.15.7:0/4074997323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e942e93f0 con 0x560e9467b670 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1faf400640 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8035000 tx=0x7f1fa8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0e1459847e6e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1f9aa00640 1 -- 172.21.15.7:0/4074997323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8033860 con 0x560e9467b670 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1f9aa00640 1 -- 172.21.15.7:0/4074997323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8051070 con 0x560e9467b670 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1f9aa00640 1 -- 172.21.15.7:0/4074997323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8056470 con 0x560e9467b670 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1fb826e740 1 -- 172.21.15.7:0/4074997323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 msgr2=0x560e946777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.456+0000 7f1fb826e740 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8035000 tx=0x7f1fa8041000 comp rx=0 tx=0).stop 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 -- 172.21.15.7:0/4074997323 shutdown_connections 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672a90 0x560e94687320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e946777e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 --2- 172.21.15.7:0/4074997323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672170 0x560e9467adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 -- 172.21.15.7:0/4074997323 >> 172.21.15.7:0/4074997323 conn(0x560e9466c8a0 msgr2=0x560e94678640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 -- 172.21.15.7:0/4074997323 shutdown_connections 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 -- 172.21.15.7:0/4074997323 wait complete. 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 -- start start 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.457+0000 7f1fb826e740 1 Processor -- start 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94683d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e94674ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b2630 con 0x560e9467b670 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b3aa0 con 0x560e94672170 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b2d00 con 0x560e94672a90 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fb826e740 10 client.0 _ll_register_callbacks cb 0x7f1fa0011ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1faea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1faea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39046/0 (socket says 172.21.15.7:39046) 2024-04-20T21:36:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1faea00640 1 -- 172.21.15.7:0/2409581920 learned_addr learned my addr 172.21.15.7:0/2409581920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1fae000640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94683d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.458+0000 7f1faf400640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e94674ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 -- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 msgr2=0x560e94683d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94683d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 -- 172.21.15.7:0/2409581920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 msgr2=0x560e94674ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e94674ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 -- 172.21.15.7:0/2409581920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e943b37d0 con 0x560e94672170 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faf400640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e94674ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1faea00640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006010 tx=0x7f1fa0011940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765dde9c400e3066 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1f8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.459+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0028040 con 0x560e94672170 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa8051070 con 0x560e94672170 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1fa000f690 con 0x560e94672170 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1fa0012930 con 0x560e94672170 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1fb826e740 10 client.26030 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:51.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e946f11b0 con 0x560e94672170 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.460+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8051070 con 0x560e94672170 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f1fa003d020 con 0x560e94672170 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 10 client.26030 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 10 client.26030 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 20 client.26030 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e946f18e0 0x560e946f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 4 client.26030 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e945de8e0 con 0x560e946f18e0 2024-04-20T21:36:51.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.461+0000 7f1fae000640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e946f18e0 0x560e946f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1f8fe00640 10 client.26030 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:51.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fae000640 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e946f18e0 0x560e946f3d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f351bec8aa458b2d server_cookie=a8253b18a803583f in_seq=0 out_seq=0 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1f8fe00640 1 -- 172.21.15.7:0/2409581920 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e945de8e0 con 0x560e946f18e0 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1f8fe00640 10 client.26030 handle_command_reply: tid=0 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 client.26030 shutdown 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 msgr2=0x560e94683830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006010 tx=0x7f1fa0011940 comp rx=0 tx=0).stop 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 shutdown_connections 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560e946f18e0 0x560e946f3d00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e9467b670 0x560e94674ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e94672a90 0x560e94683d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 --2- 172.21.15.7:0/2409581920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e94672170 0x560e94683830 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 >> 172.21.15.7:0/2409581920 conn(0x560e9466c8a0 msgr2=0x560e9466b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1f8fe00640 0 client.26030 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:36:51.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 shutdown_connections 2024-04-20T21:36:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 1 -- 172.21.15.7:0/2409581920 wait complete. 2024-04-20T21:36:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.463+0000 7f1fb826e740 20 client.26030 trim_cache size 0 max 16384 2024-04-20T21:36:51.465 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:51.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 msgr2=0x7f1f940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 0x7f1f940a61e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80099a0 tx=0x7f1fa8005fd0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 msgr2=0x7f1fb01a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00e9f0 tx=0x7f1f9c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 shutdown_connections 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1fb010e3e0 0x7f1fb019ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1fb0108e40 0x7f1fb01a4da0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1f940a3d20 0x7f1f940a61e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 --2- 172.21.15.7:0/2345813575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fb01084a0 0x7f1fb01a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 >> 172.21.15.7:0/2345813575 conn(0x7f1fb0077d70 msgr2=0x7f1fb0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 shutdown_connections 2024-04-20T21:36:51.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.464+0000 7f1fafe00640 1 -- 172.21.15.7:0/2345813575 wait complete. 2024-04-20T21:36:51.487 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:51 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/755980511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:51.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:36:51.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/755980511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:51.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:51 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/755980511' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:36:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.912+0000 7f7b7be00640 1 -- 172.21.15.7:0/646837760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740a5b50 msgr2=0x7f7b740a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.912+0000 7f7b7be00640 1 --2- 172.21.15.7:0/646837760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740a5b50 0x7f7b740a9b90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7b680074e0 tx=0x7f7b6802c8d0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.913+0000 7f7b7be00640 1 -- 172.21.15.7:0/646837760 shutdown_connections 2024-04-20T21:36:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.913+0000 7f7b7be00640 1 --2- 172.21.15.7:0/646837760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740a5b50 0x7f7b740a9b90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.913+0000 7f7b7be00640 1 --2- 172.21.15.7:0/646837760 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740aa270 0x7f7b740b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.913+0000 7f7b7be00640 1 --2- 172.21.15.7:0/646837760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b740a51a0 0x7f7b740a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.913+0000 7f7b7be00640 1 -- 172.21.15.7:0/646837760 >> 172.21.15.7:0/646837760 conn(0x7f7b7401a640 msgr2=0x7f7b7401aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.914+0000 7f7b7be00640 1 -- 172.21.15.7:0/646837760 shutdown_connections 2024-04-20T21:36:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.914+0000 7f7b7be00640 1 -- 172.21.15.7:0/646837760 wait complete. 2024-04-20T21:36:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.915+0000 7f7b7be00640 1 Processor -- start 2024-04-20T21:36:51.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.916+0000 7f7b7be00640 1 -- start start 2024-04-20T21:36:51.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740aa270 0x7f7b7413b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b7413fbc0 0x7f7b74140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52260/0 (socket says 172.21.15.7:52260) 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b4e90 con 0x7f7b7413fbc0 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7aa00640 1 -- 172.21.15.7:0/3214053642 learned_addr learned my addr 172.21.15.7:0/3214053642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b4d10 con 0x7f7b740aa270 2024-04-20T21:36:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.917+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7b740b5010 con 0x7f7b740a51a0 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.918+0000 7f7b7a000640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740aa270 0x7f7b7413b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.918+0000 7f7b7b400640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b7413fbc0 0x7f7b74140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 -- 172.21.15.7:0/3214053642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740aa270 msgr2=0x7f7b7413b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740aa270 0x7f7b7413b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 -- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b7413fbc0 msgr2=0x7f7b74140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b7413fbc0 0x7f7b74140070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741406b0 con 0x7f7b740a51a0 2024-04-20T21:36:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.919+0000 7f7b7aa00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c920 tx=0x7f7b6c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.920+0000 7f7b73400640 1 -- 172.21.15.7:0/3214053642 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c007c70 con 0x7f7b740a51a0 2024-04-20T21:36:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.920+0000 7f7b73400640 1 -- 172.21.15.7:0/3214053642 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c007e10 con 0x7f7b740a51a0 2024-04-20T21:36:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.920+0000 7f7b73400640 1 -- 172.21.15.7:0/3214053642 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00e910 con 0x7f7b740a51a0 2024-04-20T21:36:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.920+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741409a0 con 0x7f7b740a51a0 2024-04-20T21:36:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.920+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b74148160 con 0x7f7b740a51a0 2024-04-20T21:36:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.937+0000 7f7b845de740 1 Processor -- start 2024-04-20T21:36:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.940+0000 7f7b73400640 1 -- 172.21.15.7:0/3214053642 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7b6c00eab0 con 0x7f7b740a51a0 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.947+0000 7f7b845de740 1 -- start start 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3dc9050 con 0x55f5c3e6d670 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3d6f490 con 0x55f5c3e64170 2024-04-20T21:36:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.948+0000 7f7b845de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3dc5010 con 0x55f5c3e64a90 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39056/0 (socket says 172.21.15.7:39056) 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 -- 172.21.15.7:0/2408908512 learned_addr learned my addr 172.21.15.7:0/2408908512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 -- 172.21.15.7:0/2408908512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 msgr2=0x55f5c3e6d130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.949+0000 7f7b7b400640 1 -- 172.21.15.7:0/2408908512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 msgr2=0x55f5c3e6fa60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.951+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.951+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.951+0000 7f7b7b400640 1 -- 172.21.15.7:0/2408908512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3d77070 con 0x55f5c3e64170 2024-04-20T21:36:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.952+0000 7f7b73400640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 0x7f7b600aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.952+0000 7f7b73400640 1 -- 172.21.15.7:0/3214053642 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7b6c0d08c0 con 0x7f7b740a51a0 2024-04-20T21:36:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.952+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c067c40 tx=0x7f7b7c050730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23a8bb9026c18f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.953+0000 7f7b7aa00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 0x7f7b600aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.953+0000 7f7b67e00640 1 -- 172.21.15.7:0/2408908512 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c07a070 con 0x55f5c3e64170 2024-04-20T21:36:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.953+0000 7f7b67e00640 1 -- 172.21.15.7:0/2408908512 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c076440 con 0x55f5c3e64170 2024-04-20T21:36:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.953+0000 7f7b67e00640 1 -- 172.21.15.7:0/2408908512 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c075630 con 0x55f5c3e64170 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.954+0000 7f7b845de740 1 -- 172.21.15.7:0/2408908512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 msgr2=0x55f5c3e64550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.954+0000 7f7b845de740 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c067c40 tx=0x7f7b7c050730 comp rx=0 tx=0).stop 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.954+0000 7f7b7aa00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 0x7f7b600aa740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55f5c3e63c40 tx=0x7f7b6c00e3b0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.955+0000 7f7b845de740 1 -- 172.21.15.7:0/2408908512 shutdown_connections 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.955+0000 7f7b845de740 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.955+0000 7f7b845de740 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.955+0000 7f7b845de740 1 --2- 172.21.15.7:0/2408908512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e64550 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.955+0000 7f7b845de740 1 -- 172.21.15.7:0/2408908512 >> 172.21.15.7:0/2408908512 conn(0x55f5c3e5e8a0 msgr2=0x55f5c3e5ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.957+0000 7f7b845de740 1 -- 172.21.15.7:0/2408908512 shutdown_connections 2024-04-20T21:36:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.957+0000 7f7b845de740 1 -- 172.21.15.7:0/2408908512 wait complete. 2024-04-20T21:36:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.958+0000 7f7b845de740 1 -- start start 2024-04-20T21:36:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.958+0000 7f7b845de740 1 Processor -- start 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3e70460 con 0x55f5c3e64170 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3d82860 con 0x55f5c3e64a90 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f5c3e705e0 con 0x55f5c3e6d670 2024-04-20T21:36:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.959+0000 7f7b845de740 10 client.0 _ll_register_callbacks cb 0x55f5c3d50fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.960+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.960+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39062/0 (socket says 172.21.15.7:39062) 2024-04-20T21:36:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.960+0000 7f7b7a000640 1 -- 172.21.15.7:0/2273929226 learned_addr learned my addr 172.21.15.7:0/2273929226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.960+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.960+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7a000640 1 -- 172.21.15.7:0/2273929226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 msgr2=0x55f5c3e6b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7a000640 1 -- 172.21.15.7:0/2273929226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 msgr2=0x55f5c3e6a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7a000640 1 -- 172.21.15.7:0/2273929226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3d86950 con 0x55f5c3e64a90 2024-04-20T21:36:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.962+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.963+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b680074e0 tx=0x7f7b68007cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f96490d48d326ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.963+0000 7f7b64c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:51.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.966+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004250 con 0x55f5c3e64a90 2024-04-20T21:36:51.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.966+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b7c076440 con 0x55f5c3e64a90 2024-04-20T21:36:51.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.967+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7b68037040 con 0x55f5c3e64a90 2024-04-20T21:36:51.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.968+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b68004250 con 0x55f5c3e64a90 2024-04-20T21:36:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.968+0000 7f7b845de740 10 client.26042 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.968+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5c3d54820 con 0x55f5c3e64a90 2024-04-20T21:36:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.969+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b7c076440 con 0x55f5c3e64a90 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.970+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f7b680035f0 con 0x55f5c3e64a90 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.970+0000 7f7b845de740 10 client.26042 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.970+0000 7f7b845de740 10 client.26042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.970+0000 7f7b845de740 20 client.26042 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.971+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3e7d730 0x55f5c3e7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.971+0000 7f7b845de740 4 client.26042 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.971+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c3dd6a70 con 0x55f5c3e7d730 2024-04-20T21:36:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.972+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3e7d730 0x55f5c3e7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.973+0000 7f7b64c00640 10 client.26042 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.973+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3e7d730 0x55f5c3e7fb60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde73e5a59c3b95b server_cookie=e64dad96d24bef5f in_seq=0 out_seq=0 2024-04-20T21:36:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.977+0000 7f7b64c00640 1 -- 172.21.15.7:0/2273929226 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5c3dd6a70 con 0x55f5c3e7d730 2024-04-20T21:36:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.977+0000 7f7b64c00640 10 client.26042 handle_command_reply: tid=0 2024-04-20T21:36:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.977+0000 7f7b845de740 1 client.26042 shutdown 2024-04-20T21:36:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 msgr2=0x55f5c3e6acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b680074e0 tx=0x7f7b68007cf0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 shutdown_connections 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e6acb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3e7d730 0x55f5c3e7fb60 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e6b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 --2- 172.21.15.7:0/2273929226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 >> 172.21.15.7:0/2273929226 conn(0x55f5c3e5e8a0 msgr2=0x55f5c3e605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.979+0000 7f7b64c00640 0 client.26042 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:51.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.981+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 shutdown_connections 2024-04-20T21:36:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.981+0000 7f7b845de740 1 -- 172.21.15.7:0/2273929226 wait complete. 2024-04-20T21:36:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:51.981+0000 7f7b845de740 20 client.26042 trim_cache size 0 max 16384 2024-04-20T21:36:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.013+0000 7f7b845de740 1 Processor -- start 2024-04-20T21:36:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.013+0000 7f7b845de740 1 -- start start 2024-04-20T21:36:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.013+0000 7f7b845de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.013+0000 7f7b845de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e79320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.014+0000 7f7b845de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.014+0000 7f7b845de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74152ee0 con 0x55f5c3e64170 2024-04-20T21:36:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.014+0000 7f7b845de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741507a0 con 0x55f5c3e64a90 2024-04-20T21:36:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.014+0000 7f7b845de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7414e5f0 con 0x55f5c3e6d670 2024-04-20T21:36:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34278/0 (socket says 172.21.15.7:34278) 2024-04-20T21:36:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7b400640 1 -- 172.21.15.7:0/2563272055 learned_addr learned my addr 172.21.15.7:0/2563272055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e79320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7a000640 1 -- 172.21.15.7:0/2563272055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 msgr2=0x55f5c3e85470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.016+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e85470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.017+0000 7f7b7a000640 1 -- 172.21.15.7:0/2563272055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 msgr2=0x55f5c3e6cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.017+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.017+0000 7f7b7a000640 1 -- 172.21.15.7:0/2563272055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c3e6fdd0 con 0x55f5c3e64a90 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.017+0000 7f7b7b400640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.018+0000 7f7b7a000640 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e79320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55f5c3e6bd40 tx=0x7f7b6800ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda228f4abd8ba10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.018+0000 7f7b67400640 1 -- 172.21.15.7:0/2563272055 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68003be0 con 0x55f5c3e64a90 2024-04-20T21:36:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.018+0000 7f7b67400640 1 -- 172.21.15.7:0/2563272055 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68041070 con 0x55f5c3e64a90 2024-04-20T21:36:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.019+0000 7f7b845de740 1 -- 172.21.15.7:0/2563272055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 msgr2=0x55f5c3e79320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b67400640 1 -- 172.21.15.7:0/2563272055 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68004a20 con 0x55f5c3e64a90 2024-04-20T21:36:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e79320 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55f5c3e6bd40 tx=0x7f7b6800ac00 comp rx=0 tx=0).stop 2024-04-20T21:36:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 -- 172.21.15.7:0/2563272055 shutdown_connections 2024-04-20T21:36:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64a90 0x55f5c3e79320 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e6d670 0x55f5c3e85470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 --2- 172.21.15.7:0/2563272055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e64170 0x55f5c3e6cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.020+0000 7f7b845de740 1 -- 172.21.15.7:0/2563272055 >> 172.21.15.7:0/2563272055 conn(0x55f5c3e5e8a0 msgr2=0x55f5c3e5d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.021+0000 7f7b845de740 1 -- 172.21.15.7:0/2563272055 shutdown_connections 2024-04-20T21:36:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.021+0000 7f7b845de740 1 -- 172.21.15.7:0/2563272055 wait complete. 2024-04-20T21:36:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.021+0000 7f7b845de740 1 -- start start 2024-04-20T21:36:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.022+0000 7f7b845de740 1 Processor -- start 2024-04-20T21:36:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.022+0000 7f7b845de740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e67b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e73ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7414ec60 con 0x55f5c3e6d670 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741500d0 con 0x55f5c3e64170 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7414f330 con 0x55f5c3e64a90 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b845de740 10 client.0 _ll_register_callbacks cb 0x7f7b68005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:52272/0 (socket says 172.21.15.7:52272) 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.023+0000 7f7b7a000640 1 -- 172.21.15.7:0/935758934 learned_addr learned my addr 172.21.15.7:0/935758934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.024+0000 7f7b7b400640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e67b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7a000640 1 -- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 msgr2=0x55f5c3e67b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:52.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7a000640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e67b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7a000640 1 -- 172.21.15.7:0/935758934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 msgr2=0x55f5c3e73ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7b400640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e73ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7a000640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e73ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7a000640 1 -- 172.21.15.7:0/935758934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c3adb3f0 con 0x55f5c3e64a90 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.025+0000 7f7b7b400640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e73ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.026+0000 7f7b64c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.026+0000 7f7b7a000640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802ce10 tx=0x7f7b68005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f791a08d155ec9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.026+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b680048c0 con 0x55f5c3e64a90 2024-04-20T21:36:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.026+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b68041070 con 0x55f5c3e64a90 2024-04-20T21:36:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.027+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7b680046c0 con 0x55f5c3e64a90 2024-04-20T21:36:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.028+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b680048c0 con 0x55f5c3e64a90 2024-04-20T21:36:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.029+0000 7f7b845de740 10 client.26026 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.029+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f5c3ba57d0 con 0x55f5c3e64a90 2024-04-20T21:36:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.030+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b68041070 con 0x55f5c3e64a90 2024-04-20T21:36:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.030+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f7b68057440 con 0x55f5c3e64a90 2024-04-20T21:36:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.032+0000 7f7b845de740 10 client.26026 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.032+0000 7f7b845de740 10 client.26026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.035+0000 7f7b845de740 20 client.26026 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.035+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3eecf50 0x55f5c3eef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.035+0000 7f7b845de740 4 client.26026 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:36:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.035+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5c3dd08e0 con 0x55f5c3eecf50 2024-04-20T21:36:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.035+0000 7f7b7b400640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3eecf50 0x55f5c3eef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.040+0000 7f7b64c00640 10 client.26026 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.040+0000 7f7b7b400640 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3eecf50 0x55f5c3eef370 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7dea6b955c7e21 server_cookie=f105b5781786a60d in_seq=0 out_seq=0 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.040+0000 7f7b64c00640 1 -- 172.21.15.7:0/935758934 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5c3dd08e0 con 0x55f5c3eecf50 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.040+0000 7f7b64c00640 10 client.26026 handle_command_reply: tid=0 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.040+0000 7f7b845de740 1 client.26026 shutdown 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 msgr2=0x55f5c3e68040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:52.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b6802ce10 tx=0x7f7b68005fd0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 shutdown_connections 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f5c3eecf50 0x55f5c3eef370 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f5c3e6d670 0x55f5c3e73ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b64c00640 0 client.26026 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f5c3e64a90 0x55f5c3e68040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 --2- 172.21.15.7:0/935758934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5c3e64170 0x55f5c3e67b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.041+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 >> 172.21.15.7:0/935758934 conn(0x55f5c3e5e8a0 msgr2=0x55f5c3e5fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.045+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 shutdown_connections 2024-04-20T21:36:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.045+0000 7f7b845de740 1 -- 172.21.15.7:0/935758934 wait complete. 2024-04-20T21:36:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.045+0000 7f7b845de740 20 client.26026 trim_cache size 0 max 16384 2024-04-20T21:36:52.044 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:36:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.047+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 msgr2=0x7f7b600aa740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.047+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 0x7f7b600aa740 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55f5c3e63c40 tx=0x7f7b6c00e3b0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.047+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 msgr2=0x7f7b7413b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.047+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c920 tx=0x7f7b6c00cde0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 shutdown_connections 2024-04-20T21:36:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7b600a8280 0x7f7b600aa740 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b740aa270 0x7f7b7413b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b7413fbc0 0x7f7b74140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 --2- 172.21.15.7:0/3214053642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7b740a51a0 0x7f7b7413b350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 >> 172.21.15.7:0/3214053642 conn(0x7f7b7401a640 msgr2=0x7f7b740a4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.052+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 shutdown_connections 2024-04-20T21:36:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:52.053+0000 7f7b7be00640 1 -- 172.21.15.7:0/3214053642 wait complete. 2024-04-20T21:36:52.348 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:52 smithi007 ceph-mon[16867]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-20T21:36:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:52 smithi016 ceph-mon[21409]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-20T21:36:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:52 smithi186 ceph-mon[22157]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 100 MiB/s wr, 14.99k op/s 2024-04-20T21:36:53.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:36:53.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:36:53.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:36:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.489+0000 7fbbce800640 1 -- 172.21.15.7:0/4270938399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074330 msgr2=0x7fbbc8074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.489+0000 7fbbce800640 1 --2- 172.21.15.7:0/4270938399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074330 0x7fbbc8074710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009900 tx=0x7fbbbc02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.490+0000 7fbbce800640 1 -- 172.21.15.7:0/4270938399 shutdown_connections 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.490+0000 7fbbce800640 1 --2- 172.21.15.7:0/4270938399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 0x7fbbc8110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.490+0000 7fbbce800640 1 --2- 172.21.15.7:0/4270938399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074c50 0x7fbbc810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.490+0000 7fbbce800640 1 --2- 172.21.15.7:0/4270938399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074330 0x7fbbc8074710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.490+0000 7fbbce800640 1 -- 172.21.15.7:0/4270938399 >> 172.21.15.7:0/4270938399 conn(0x7fbbc80fd8b0 msgr2=0x7fbbc80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.491+0000 7fbbce800640 1 -- 172.21.15.7:0/4270938399 shutdown_connections 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.491+0000 7fbbce800640 1 -- 172.21.15.7:0/4270938399 wait complete. 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.493+0000 7fbbce800640 1 Processor -- start 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.494+0000 7fbbce800640 1 -- start start 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074330 0x7fbbc810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 0x7fbbc810b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81143c0 con 0x7fbbc8074330 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8114240 con 0x7fbbc810e600 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbce800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8114540 con 0x7fbbc8074c50 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbcca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbcde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 0x7fbbc810b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbcca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45344/0 (socket says 172.21.15.7:45344) 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbcca00640 1 -- 172.21.15.7:0/3784504253 learned_addr learned my addr 172.21.15.7:0/3784504253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.496+0000 7fbbcd400640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074330 0x7fbbc810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.497+0000 7fbbcca00640 1 -- 172.21.15.7:0/3784504253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 msgr2=0x7fbbc810b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.497+0000 7fbbcca00640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 0x7fbbc810b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.497+0000 7fbbcca00640 1 -- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074330 msgr2=0x7fbbc810df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.497+0000 7fbbcca00640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074330 0x7fbbc810df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.497+0000 7fbbcca00640 1 -- 172.21.15.7:0/3784504253 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc810be40 con 0x7fbbc8074c50 2024-04-20T21:36:53.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.502+0000 7fbbcca00640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c890 tx=0x7fbbb400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.504+0000 7fbbc6000640 1 -- 172.21.15.7:0/3784504253 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4004220 con 0x7fbbc8074c50 2024-04-20T21:36:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.504+0000 7fbbc6000640 1 -- 172.21.15.7:0/3784504253 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb40043c0 con 0x7fbbc8074c50 2024-04-20T21:36:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.504+0000 7fbbc6000640 1 -- 172.21.15.7:0/3784504253 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40049e0 con 0x7fbbc8074c50 2024-04-20T21:36:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.504+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81ab220 con 0x7fbbc8074c50 2024-04-20T21:36:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.505+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81ab720 con 0x7fbbc8074c50 2024-04-20T21:36:53.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.516+0000 7fbbc6000640 1 -- 172.21.15.7:0/3784504253 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbbb40079b0 con 0x7fbbc8074c50 2024-04-20T21:36:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.519+0000 7fbbc6000640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 0x7fbbb00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.519+0000 7fbbd2a42740 1 Processor -- start 2024-04-20T21:36:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.519+0000 7fbbc6000640 1 -- 172.21.15.7:0/3784504253 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbbb40d0720 con 0x7fbbc8074c50 2024-04-20T21:36:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.520+0000 7fbbd2a42740 1 -- start start 2024-04-20T21:36:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.521+0000 7fbbcd400640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 0x7fbbb00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f7250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 0x55f21f7279e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f21f680d20 con 0x55f21f7255f0 2024-04-20T21:36:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f21f6273e0 con 0x55f21f71c0f0 2024-04-20T21:36:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f21f67cce0 con 0x55f21f71ca10 2024-04-20T21:36:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44970/0 (socket says 172.21.15.7:44970) 2024-04-20T21:36:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.522+0000 7fbbcd400640 1 -- 172.21.15.7:0/1550810326 learned_addr learned my addr 172.21.15.7:0/1550810326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.523+0000 7fbbcde00640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 0x55f21f7279e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcca00640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f7250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 -- 172.21.15.7:0/1550810326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 msgr2=0x55f21f7250b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f7250b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 -- 172.21.15.7:0/1550810326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 msgr2=0x55f21f7279e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 0x55f21f7279e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 -- 172.21.15.7:0/1550810326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21f646980 con 0x55f21f71c0f0 2024-04-20T21:36:53.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcde00640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 0x55f21f7279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcca00640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f7250b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.524+0000 7fbbcd400640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 0x7fbbb00a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc000c00 tx=0x7fbbbc004280 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.525+0000 7fbbcd400640 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0096c0 tx=0x7fbbbc04a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6901b5caba86dffd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.525+0000 7fbbba600640 1 -- 172.21.15.7:0/1550810326 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0046c0 con 0x55f21f71c0f0 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.526+0000 7fbbba600640 1 -- 172.21.15.7:0/1550810326 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbbc004f00 con 0x55f21f71c0f0 2024-04-20T21:36:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.526+0000 7fbbba600640 1 -- 172.21.15.7:0/1550810326 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc0038e0 con 0x55f21f71c0f0 2024-04-20T21:36:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 -- 172.21.15.7:0/1550810326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 msgr2=0x55f21f71c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0096c0 tx=0x7fbbbc04a000 comp rx=0 tx=0).stop 2024-04-20T21:36:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 -- 172.21.15.7:0/1550810326 shutdown_connections 2024-04-20T21:36:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f7255f0 0x55f21f7279e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f7250b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/1550810326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f71c4d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.527+0000 7fbbd2a42740 1 -- 172.21.15.7:0/1550810326 >> 172.21.15.7:0/1550810326 conn(0x55f21f716820 msgr2=0x55f21f716c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.529+0000 7fbbd2a42740 1 -- 172.21.15.7:0/1550810326 shutdown_connections 2024-04-20T21:36:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.529+0000 7fbbd2a42740 1 -- 172.21.15.7:0/1550810326 wait complete. 2024-04-20T21:36:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.531+0000 7fbbd2a42740 1 -- start start 2024-04-20T21:36:53.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.531+0000 7fbbd2a42740 1 Processor -- start 2024-04-20T21:36:53.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.533+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f7226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.533+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71ca10 0x55f21f722bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.533+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 0x55f21f723130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.534+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f7226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.534+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f7226b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44986/0 (socket says 172.21.15.7:44986) 2024-04-20T21:36:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.535+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f21f728320 con 0x55f21f71ca10 2024-04-20T21:36:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.535+0000 7fbbcd400640 1 -- 172.21.15.7:0/2948299711 learned_addr learned my addr 172.21.15.7:0/2948299711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.535+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f21f63a7a0 con 0x55f21f71c0f0 2024-04-20T21:36:53.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.535+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f21f7284a0 con 0x55f21f7255f0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.535+0000 7fbbcde00640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 0x55f21f723130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcca00640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71ca10 0x55f21f722bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbd2a42740 10 client.0 _ll_register_callbacks cb 0x55f21f63cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 -- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 msgr2=0x55f21f7226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f7226b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 -- 172.21.15.7:0/2948299711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71ca10 msgr2=0x55f21f722bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71ca10 0x55f21f722bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21f45b7e0 con 0x55f21f7255f0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc004f00 con 0x55f21f7255f0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbcde00640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 0x55f21f723130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbbc81013d0 tx=0x7fbbc0012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcae4097356f0106 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.536+0000 7fbbaea00640 10 client.26035 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.537+0000 7fbbaea00640 1 -- 172.21.15.7:0/2948299711 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0010df0 con 0x55f21f7255f0 2024-04-20T21:36:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.537+0000 7fbbaea00640 1 -- 172.21.15.7:0/2948299711 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbbc0003970 con 0x55f21f7255f0 2024-04-20T21:36:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.537+0000 7fbbaea00640 1 -- 172.21.15.7:0/2948299711 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbc0003bb0 con 0x55f21f7255f0 2024-04-20T21:36:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.537+0000 7fbbd2a42740 10 client.26035 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:53.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55f21f4bbf70 con 0x55f21f7255f0 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbaea00640 1 -- 172.21.15.7:0/2948299711 <== mon.1 v2:172.21.15.186:3300/0 4 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fbbc0019030 con 0x55f21f7255f0 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 10 client.26035 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 20 client.26035 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f735730 0x55f21f737b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 4 client.26035 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.538+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21f68e740 con 0x55f21f735730 2024-04-20T21:36:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.539+0000 7fbbcd400640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f735730 0x55f21f737b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.540+0000 7fbbcd400640 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f735730 0x55f21f737b80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7bcd5b3fe04361 server_cookie=a69bcd1de3021b1d in_seq=0 out_seq=0 2024-04-20T21:36:53.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.540+0000 7fbbaea00640 10 client.26035 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.541+0000 7fbbaea00640 1 -- 172.21.15.7:0/2948299711 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f21f68e740 con 0x55f21f735730 2024-04-20T21:36:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.541+0000 7fbbaea00640 10 client.26035 handle_command_reply: tid=0 2024-04-20T21:36:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.541+0000 7fbbd2a42740 1 client.26035 shutdown 2024-04-20T21:36:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.542+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 msgr2=0x55f21f723130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.542+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 0x55f21f723130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbbc81013d0 tx=0x7fbbc0012b80 comp rx=0 tx=0).stop 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 shutdown_connections 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f735730 0x55f21f737b80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f7255f0 0x55f21f723130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71ca10 0x55f21f722bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/2948299711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f71c0f0 0x55f21f7226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 >> 172.21.15.7:0/2948299711 conn(0x55f21f716820 msgr2=0x55f21f718570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbaea00640 0 client.26035 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 shutdown_connections 2024-04-20T21:36:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 1 -- 172.21.15.7:0/2948299711 wait complete. 2024-04-20T21:36:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.543+0000 7fbbd2a42740 20 client.26035 trim_cache size 0 max 16384 2024-04-20T21:36:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.590+0000 7fbbd2a42740 1 Processor -- start 2024-04-20T21:36:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 -- start start 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f724de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f731230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f721830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b6a00 con 0x55f21f71c0f0 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b42c0 con 0x55f21f7255f0 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.591+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b2110 con 0x55f21f71ca10 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f724de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f724de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41872/0 (socket says 172.21.15.7:41872) 2024-04-20T21:36:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcd400640 1 -- 172.21.15.7:0/4274382564 learned_addr learned my addr 172.21.15.7:0/4274382564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f731230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 -- 172.21.15.7:0/4274382564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 msgr2=0x55f21f721830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f721830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 -- 172.21.15.7:0/4274382564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 msgr2=0x55f21f724de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f724de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.592+0000 7fbbcca00640 1 -- 172.21.15.7:0/4274382564 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21f727d50 con 0x55f21f71ca10 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.593+0000 7fbbcca00640 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f731230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40cdec0 tx=0x7fbbb409c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5c4db90c762f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.593+0000 7fbbb9c00640 1 -- 172.21.15.7:0/4274382564 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb401d070 con 0x55f21f71ca10 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.593+0000 7fbbb9c00640 1 -- 172.21.15.7:0/4274382564 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb40a0ac0 con 0x55f21f71ca10 2024-04-20T21:36:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.593+0000 7fbbb9c00640 1 -- 172.21.15.7:0/4274382564 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007ca0 con 0x55f21f71ca10 2024-04-20T21:36:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.594+0000 7fbbd2a42740 1 -- 172.21.15.7:0/4274382564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 msgr2=0x55f21f731230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.594+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f731230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40cdec0 tx=0x7fbbb409c300 comp rx=0 tx=0).stop 2024-04-20T21:36:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 -- 172.21.15.7:0/4274382564 shutdown_connections 2024-04-20T21:36:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f721830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f731230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/4274382564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f724de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 -- 172.21.15.7:0/4274382564 >> 172.21.15.7:0/4274382564 conn(0x55f21f716820 msgr2=0x55f21f715170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.595+0000 7fbbd2a42740 1 -- 172.21.15.7:0/4274382564 shutdown_connections 2024-04-20T21:36:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.596+0000 7fbbd2a42740 1 -- 172.21.15.7:0/4274382564 wait complete. 2024-04-20T21:36:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.596+0000 7fbbd2a42740 1 -- start start 2024-04-20T21:36:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.597+0000 7fbbd2a42740 1 Processor -- start 2024-04-20T21:36:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.598+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f71f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.598+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f71fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbd2a42740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b2780 con 0x55f21f71c0f0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b3bf0 con 0x55f21f7255f0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbd2a42740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81b2e50 con 0x55f21f71ca10 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbd2a42740 10 client.0 _ll_register_callbacks cb 0x7fbbc0009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44994/0 (socket says 172.21.15.7:44994) 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 -- 172.21.15.7:0/310083093 learned_addr learned my addr 172.21.15.7:0/310083093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcd400640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f71f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 -- 172.21.15.7:0/310083093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 msgr2=0x55f21f71fc30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f71fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 -- 172.21.15.7:0/310083093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 msgr2=0x55f21f71f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f71f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.599+0000 7fbbcde00640 1 -- 172.21.15.7:0/310083093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21f353170 con 0x55f21f7255f0 2024-04-20T21:36:53.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.600+0000 7fbbcd400640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f71f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.600+0000 7fbbaea00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:36:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.600+0000 7fbbcde00640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0012bc0 tx=0x7fbbc0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa1c1d412f1ccc1f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.601+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0009600 con 0x55f21f7255f0 2024-04-20T21:36:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.601+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbbc0004d30 con 0x55f21f7255f0 2024-04-20T21:36:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.601+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb40a0ac0 con 0x55f21f7255f0 2024-04-20T21:36:53.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.602+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbbc001a4c0 con 0x55f21f7255f0 2024-04-20T21:36:53.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.602+0000 7fbbd2a42740 10 client.26060 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:53.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.602+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f21f79c130 con 0x55f21f7255f0 2024-04-20T21:36:53.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbb40a0ac0 con 0x55f21f7255f0 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7fbbc0019070 con 0x55f21f7255f0 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 10 client.26060 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 10 client.26060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 20 client.26060 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f79c8c0 0x55f21f79ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 4 client.26060 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:36:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.603+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f21f737f10 con 0x55f21f79c8c0 2024-04-20T21:36:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.604+0000 7fbbcd400640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f79c8c0 0x55f21f79ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.605+0000 7fbbcd400640 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f79c8c0 0x55f21f79ece0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f2997c48ab178b server_cookie=58b9d0f7bba6b14c in_seq=0 out_seq=0 2024-04-20T21:36:53.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.605+0000 7fbbaea00640 10 client.26060 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.606+0000 7fbbaea00640 1 -- 172.21.15.7:0/310083093 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f21f737f10 con 0x55f21f79c8c0 2024-04-20T21:36:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.606+0000 7fbbaea00640 10 client.26060 handle_command_reply: tid=0 2024-04-20T21:36:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 client.26060 shutdown 2024-04-20T21:36:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 msgr2=0x55f21f79acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0012bc0 tx=0x7fbbc0004ab0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 shutdown_connections 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f21f7255f0 0x55f21f79acc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f21f79c8c0 0x55f21f79ece0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f21f71ca10 0x55f21f71fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 --2- 172.21.15.7:0/310083093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f21f71c0f0 0x55f21f71f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.607+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 >> 172.21.15.7:0/310083093 conn(0x55f21f716820 msgr2=0x55f21f7216b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.608+0000 7fbbaea00640 0 client.26060 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.608+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 shutdown_connections 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.608+0000 7fbbd2a42740 1 -- 172.21.15.7:0/310083093 wait complete. 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.608+0000 7fbbd2a42740 20 client.26060 trim_cache size 0 max 16384 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "755ad1ed-c483-498c-98fd-b6fd781a7d3d", 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 msgr2=0x7fbbb00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 0x7fbbb00a6260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc000c00 tx=0x7fbbbc004280 comp rx=0 tx=0).stop 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 msgr2=0x7fbbc810b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400c890 tx=0x7fbbb400cd50 comp rx=0 tx=0).stop 2024-04-20T21:36:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 shutdown_connections 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbbb00a3da0 0x7fbbb00a6260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbc810e600 0x7fbbc810b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbbc8074c50 0x7fbbc810b020 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 --2- 172.21.15.7:0/3784504253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbc8074330 0x7fbbc810df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 >> 172.21.15.7:0/3784504253 conn(0x7fbbc80fd8b0 msgr2=0x7fbbc80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 shutdown_connections 2024-04-20T21:36:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.609+0000 7fbbce800640 1 -- 172.21.15.7:0/3784504253 wait complete. 2024-04-20T21:36:53.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:36:53.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:36:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.987+0000 7f6f79200640 1 -- 172.21.15.7:0/3426113076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0a5710 msgr2=0x7f6f6c0b5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.987+0000 7f6f72000640 1 -- 172.21.15.7:0/3426113076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7409b070 con 0x7f6f6c0a5710 2024-04-20T21:36:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.987+0000 7f6f79200640 1 --2- 172.21.15.7:0/3426113076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0b5e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6f74067dd0 tx=0x7f6f7408f8b0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.988+0000 7f6f79200640 1 -- 172.21.15.7:0/3426113076 shutdown_connections 2024-04-20T21:36:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.988+0000 7f6f79200640 1 --2- 172.21.15.7:0/3426113076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a4d50 0x7f6f6c0a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.988+0000 7f6f79200640 1 --2- 172.21.15.7:0/3426113076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0b5e00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.988+0000 7f6f79200640 1 --2- 172.21.15.7:0/3426113076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0af0e0 0x7f6f6c0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.988+0000 7f6f79200640 1 -- 172.21.15.7:0/3426113076 >> 172.21.15.7:0/3426113076 conn(0x7f6f6c01a640 msgr2=0x7f6f6c01aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.989+0000 7f6f72a00640 1 --2- 172.21.15.7:0/3426113076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a4d50 0x7f6f6c0a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.990+0000 7f6f79200640 1 -- 172.21.15.7:0/3426113076 shutdown_connections 2024-04-20T21:36:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.990+0000 7f6f79200640 1 -- 172.21.15.7:0/3426113076 wait complete. 2024-04-20T21:36:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.994+0000 7f6f79200640 1 Processor -- start 2024-04-20T21:36:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.994+0000 7f6f79200640 1 -- start start 2024-04-20T21:36:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.995+0000 7f6f79200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.995+0000 7f6f79200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 0x7f6f6c0aab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f79200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f79200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0b9380 con 0x7f6f6c0aa730 2024-04-20T21:36:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f79200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0b9200 con 0x7f6f6c0a5710 2024-04-20T21:36:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f79200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c0b9500 con 0x7f6f6c0ab0d0 2024-04-20T21:36:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45378/0 (socket says 172.21.15.7:45378) 2024-04-20T21:36:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.996+0000 7f6f73e00640 1 -- 172.21.15.7:0/764370653 learned_addr learned my addr 172.21.15.7:0/764370653 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f72a00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 0x7f6f6c0aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f73e00640 1 -- 172.21.15.7:0/764370653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a5710 msgr2=0x7f6f6c0ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f73e00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f73e00640 1 -- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 msgr2=0x7f6f6c0aab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f73e00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 0x7f6f6c0aab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f73e00640 1 -- 172.21.15.7:0/764370653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c145de0 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.997+0000 7f6f72a00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 0x7f6f6c0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.998+0000 7f6f73e00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6f74067750 tx=0x7f6f7408fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.998+0000 7f6f67e00640 1 -- 172.21.15.7:0/764370653 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740a4070 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.999+0000 7f6f67e00640 1 -- 172.21.15.7:0/764370653 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f74068a60 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:53.999+0000 7f6f67e00640 1 -- 172.21.15.7:0/764370653 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7409b040 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.000+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c146070 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.000+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c146560 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.013+0000 7f6f67e00640 1 -- 172.21.15.7:0/764370653 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6f740a7070 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.016+0000 7f6f7d627740 1 Processor -- start 2024-04-20T21:36:54.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.016+0000 7f6f7d627740 1 -- start start 2024-04-20T21:36:54.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.017+0000 7f6f67e00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 0x7f6f5c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.017+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.017+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.017+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.017+0000 7f6f67e00640 1 -- 172.21.15.7:0/764370653 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6f74155900 con 0x7f6f6c0ab0d0 2024-04-20T21:36:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f7d627740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5edc150 con 0x55e8e5f76380 2024-04-20T21:36:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f7d627740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5e82370 con 0x55e8e5f76ca0 2024-04-20T21:36:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f7d627740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5ed8110 con 0x55e8e5f7f880 2024-04-20T21:36:54.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45386/0 (socket says 172.21.15.7:45386) 2024-04-20T21:36:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 -- 172.21.15.7:0/3374708418 learned_addr learned my addr 172.21.15.7:0/3374708418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 -- 172.21.15.7:0/3374708418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 msgr2=0x55e8e5f7f340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:36:54.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7f340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 -- 172.21.15.7:0/3374708418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 msgr2=0x55e8e5f76760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f76760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.018+0000 7f6f73e00640 1 -- 172.21.15.7:0/3374708418 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e5cb87d0 con 0x55e8e5f7f880 2024-04-20T21:36:54.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.019+0000 7f6f72a00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 0x7f6f5c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.019+0000 7f6f73e00640 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f74050f60 tx=0x7f6f74067ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a7c81211e4aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.020+0000 7f6f64c00640 1 -- 172.21.15.7:0/3374708418 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74121890 con 0x55e8e5f7f880 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.020+0000 7f6f64c00640 1 -- 172.21.15.7:0/3374708418 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f74125c40 con 0x55e8e5f7f880 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 -- 172.21.15.7:0/3374708418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 msgr2=0x55e8e5f81c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f64c00640 1 -- 172.21.15.7:0/3374708418 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74121890 con 0x55e8e5f7f880 2024-04-20T21:36:54.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6f74050f60 tx=0x7f6f74067ea0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 -- 172.21.15.7:0/3374708418 shutdown_connections 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f81c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 --2- 172.21.15.7:0/3374708418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f76760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.021+0000 7f6f7d627740 1 -- 172.21.15.7:0/3374708418 >> 172.21.15.7:0/3374708418 conn(0x55e8e5f70ab0 msgr2=0x55e8e5f70e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:54.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.022+0000 7f6f72a00640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 0x7f6f5c0a62a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e8e5f75e50 tx=0x7f6f68009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.023+0000 7f6f7d627740 1 -- 172.21.15.7:0/3374708418 shutdown_connections 2024-04-20T21:36:54.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.023+0000 7f6f7d627740 1 -- 172.21.15.7:0/3374708418 wait complete. 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.024+0000 7f6f7d627740 1 -- start start 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.024+0000 7f6f7d627740 1 Processor -- start 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.025+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.025+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.025+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f7d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:45030/0 (socket says 172.21.15.7:45030) 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45390/0 (socket says 172.21.15.7:45390) 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f7d627740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5f82430 con 0x55e8e5f7f880 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f7d627740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5e95740 con 0x55e8e5f76380 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.026+0000 7f6f7d627740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e8e5f825b0 con 0x55e8e5f76ca0 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.027+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f7d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.027+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f7d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41902/0 (socket says 172.21.15.7:41902) 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.027+0000 7f6f73e00640 1 -- 172.21.15.7:0/1019538451 learned_addr learned my addr 172.21.15.7:0/1019538451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.027+0000 7f6f7d627740 10 client.0 _ll_register_callbacks cb 0x55e8e5e90b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.028+0000 7f6f72a00640 1 -- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 msgr2=0x55e8e5f7c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.028+0000 7f6f72a00640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.028+0000 7f6f72a00640 1 -- 172.21.15.7:0/1019538451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 msgr2=0x55e8e5f7d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.028+0000 7f6f72a00640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f7d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.028+0000 7f6f72a00640 1 -- 172.21.15.7:0/1019538451 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e5bee3f0 con 0x55e8e5f76ca0 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.030+0000 7f6f72a00640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c01bbf0 tx=0x7f6f68004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b8cf222d6b43ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.030+0000 7f6f59600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.031+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004220 con 0x55e8e5f76ca0 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.031+0000 7f6f73400640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.031+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f74125c40 con 0x55e8e5f76ca0 2024-04-20T21:36:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.031+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6f680043c0 con 0x55e8e5f76ca0 2024-04-20T21:36:54.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.032+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f6802b020 con 0x55e8e5f76ca0 2024-04-20T21:36:54.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.033+0000 7f6f7d627740 10 client.26047 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:54.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.033+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8e5bae0f0 con 0x55e8e5f76ca0 2024-04-20T21:36:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.034+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f74125c40 con 0x55e8e5f76ca0 2024-04-20T21:36:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.035+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f6f68030430 con 0x55e8e5f76ca0 2024-04-20T21:36:54.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.036+0000 7f6f7d627740 10 client.26047 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.036+0000 7f6f7d627740 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.037+0000 7f6f7d627740 20 client.26047 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.037+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5f8f730 0x55e8e5f91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.037+0000 7f6f7d627740 4 client.26047 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:36:54.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.037+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8e5ee9b70 con 0x55e8e5f8f730 2024-04-20T21:36:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.039+0000 7f6f73400640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5f8f730 0x55e8e5f91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.039+0000 7f6f73400640 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5f8f730 0x55e8e5f91b90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd380dd9a17aa14 server_cookie=79de024b555b1193 in_seq=0 out_seq=0 2024-04-20T21:36:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.039+0000 7f6f59600640 10 client.26047 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.040+0000 7f6f59600640 1 -- 172.21.15.7:0/1019538451 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8e5ee9b70 con 0x55e8e5f8f730 2024-04-20T21:36:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.040+0000 7f6f59600640 10 client.26047 handle_command_reply: tid=0 2024-04-20T21:36:54.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.041+0000 7f6f7d627740 1 client.26047 shutdown 2024-04-20T21:36:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 msgr2=0x55e8e5f7cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c01bbf0 tx=0x7f6f68004730 comp rx=0 tx=0).stop 2024-04-20T21:36:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 shutdown_connections 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5f8f730 0x55e8e5f91b90 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f59600640 0 client.26047 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f7d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f7cdb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 --2- 172.21.15.7:0/1019538451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76380 0x55e8e5f7c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.045+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 >> 172.21.15.7:0/1019538451 conn(0x55e8e5f70ab0 msgr2=0x55e8e5f72800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.046+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 shutdown_connections 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.046+0000 7f6f7d627740 1 -- 172.21.15.7:0/1019538451 wait complete. 2024-04-20T21:36:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.046+0000 7f6f7d627740 20 client.26047 trim_cache size 0 max 16384 2024-04-20T21:36:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.104+0000 7f6f7d627740 1 Processor -- start 2024-04-20T21:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 -- start start 2024-04-20T21:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f8b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f77e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c164740 con 0x55e8e5f76380 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c162000 con 0x55e8e5f76ca0 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.105+0000 7f6f7d627740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c15fe50 con 0x55e8e5f7f880 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f73400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f73400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41916/0 (socket says 172.21.15.7:41916) 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f73400640 1 -- 172.21.15.7:0/910533637 learned_addr learned my addr 172.21.15.7:0/910533637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:54.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f72a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f8b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f77e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.106+0000 7f6f73400640 1 -- 172.21.15.7:0/910533637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 msgr2=0x55e8e5f77e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73400640 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f77e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73400640 1 -- 172.21.15.7:0/910533637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 msgr2=0x55e8e5f8b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73400640 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f8b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73400640 1 -- 172.21.15.7:0/910533637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8e5f78490 con 0x55e8e5f76380 2024-04-20T21:36:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73e00640 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:36:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f72a00640 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f8b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.107+0000 7f6f73400640 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c01b9c0 tx=0x7f6f60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd99319eb2e3935 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.108+0000 7f6f5be00640 1 -- 172.21.15.7:0/910533637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60014110 con 0x55e8e5f76380 2024-04-20T21:36:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.108+0000 7f6f5be00640 1 -- 172.21.15.7:0/910533637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f600142b0 con 0x55e8e5f76380 2024-04-20T21:36:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.108+0000 7f6f5be00640 1 -- 172.21.15.7:0/910533637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004b60 con 0x55e8e5f76380 2024-04-20T21:36:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.109+0000 7f6f7d627740 1 -- 172.21.15.7:0/910533637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 msgr2=0x55e8e5f7b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.109+0000 7f6f7d627740 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c01b9c0 tx=0x7f6f60000bc0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 -- 172.21.15.7:0/910533637 shutdown_connections 2024-04-20T21:36:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f8b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f7f880 0x55e8e5f77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 --2- 172.21.15.7:0/910533637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f76380 0x55e8e5f7b7d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 -- 172.21.15.7:0/910533637 >> 172.21.15.7:0/910533637 conn(0x55e8e5f70ab0 msgr2=0x55e8e5f72790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 -- 172.21.15.7:0/910533637 shutdown_connections 2024-04-20T21:36:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.110+0000 7f6f7d627740 1 -- 172.21.15.7:0/910533637 wait complete. 2024-04-20T21:36:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.111+0000 7f6f7d627740 1 -- start start 2024-04-20T21:36:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.112+0000 7f6f7d627740 1 Processor -- start 2024-04-20T21:36:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f79a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f79fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1604c0 con 0x55e8e5f7f880 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c161930 con 0x55e8e5f76ca0 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c160b90 con 0x55e8e5f76380 2024-04-20T21:36:54.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f7d627740 10 client.0 _ll_register_callbacks cb 0x7f6f68009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f79fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f73400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f73400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:45410/0 (socket says 172.21.15.7:45410) 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.113+0000 7f6f73400640 1 -- 172.21.15.7:0/2956599259 learned_addr learned my addr 172.21.15.7:0/2956599259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f72a00640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f79a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73400640 1 -- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 msgr2=0x55e8e5f79a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73400640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f79a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73400640 1 -- 172.21.15.7:0/2956599259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 msgr2=0x55e8e5f79fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:36:54.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73400640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f79fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73400640 1 -- 172.21.15.7:0/2956599259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8e5f76ad0 con 0x55e8e5f76380 2024-04-20T21:36:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f72a00640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:36:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.114+0000 7f6f73e00640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.115+0000 7f6f59600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.115+0000 7f6f73400640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f60005c70 tx=0x7f6f60005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f332f790e16e8e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.115+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60014110 con 0x55e8e5f76380 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.115+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8e5ee39e0 con 0x55e8e5f76380 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.115+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6f600048d0 con 0x55e8e5f76380 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.116+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6f60004ad0 con 0x55e8e5f76380 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.116+0000 7f6f7d627740 10 client.26050 fetch_fsmap learned FSMap version 25 2024-04-20T21:36:54.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.116+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8e5ff5d40 con 0x55e8e5f76380 2024-04-20T21:36:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e8e5ee39e0 con 0x55e8e5f76380 2024-04-20T21:36:54.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 25) ==== 8347+0+0 (secure 0 0 0) 0x7f6f60012450 con 0x55e8e5f76380 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 10 client.26050 fetch_fsmap finished waiting for FSMap version 25 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 20 client.26050 populate_metadata read hostname 'smithi007' 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5ff64d0 0x55e8e5ff88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 4 client.26050 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:36:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.117+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8e5e5f450 con 0x55e8e5ff64d0 2024-04-20T21:36:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.118+0000 7f6f72a00640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5ff64d0 0x55e8e5ff88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:36:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.119+0000 7f6f59600640 10 client.26050 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:54.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.121+0000 7f6f72a00640 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5ff64d0 0x55e8e5ff88f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d966e32d7e016324 server_cookie=b893bdcbf4d70b7d in_seq=0 out_seq=0 2024-04-20T21:36:54.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.122+0000 7f6f59600640 1 -- 172.21.15.7:0/2956599259 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55e8e5e5f450 con 0x55e8e5ff64d0 2024-04-20T21:36:54.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.122+0000 7f6f59600640 10 client.26050 handle_command_reply: tid=0 2024-04-20T21:36:54.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.122+0000 7f6f7d627740 1 client.26050 shutdown 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 msgr2=0x55e8e5f79520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f60005c70 tx=0x7f6f60005250 comp rx=0 tx=0).stop 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 shutdown_connections 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e8e5f76ca0 0x55e8e5f79a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e8e5ff64d0 0x55e8e5ff88f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e8e5f7f880 0x55e8e5f79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 --2- 172.21.15.7:0/2956599259 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e8e5f76380 0x55e8e5f79520 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 >> 172.21.15.7:0/2956599259 conn(0x55e8e5f70ab0 msgr2=0x55e8e5f6f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f59600640 0 client.26050 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:36:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 shutdown_connections 2024-04-20T21:36:54.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.123+0000 7f6f7d627740 1 -- 172.21.15.7:0/2956599259 wait complete. 2024-04-20T21:36:54.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.124+0000 7f6f7d627740 20 client.26050 trim_cache size 0 max 16384 2024-04-20T21:36:54.123 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:36:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (217 inodes in the stack)", 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "755ad1ed-c483-498c-98fd-b6fd781a7d3d": { 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "755ad1ed-c483-498c-98fd-b6fd781a7d3d", 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:36:54.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.126+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 msgr2=0x7f6f5c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.126+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 0x7f6f5c0a62a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e8e5f75e50 tx=0x7f6f68009260 comp rx=0 tx=0).stop 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.126+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 msgr2=0x7f6f6c1458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.126+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6f74067750 tx=0x7f6f7408fdd0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.126+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 shutdown_connections 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f6c0ab0d0 0x7f6f6c1458a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f6c0aa730 0x7f6f6c0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6f5c0a3de0 0x7f6f5c0a62a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 --2- 172.21.15.7:0/764370653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f6c0a5710 0x7f6f6c0ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:36:54.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 >> 172.21.15.7:0/764370653 conn(0x7f6f6c01a640 msgr2=0x7f6f6c0a1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:36:54.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 shutdown_connections 2024-04-20T21:36:54.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:36:54.127+0000 7f6f79200640 1 -- 172.21.15.7:0/764370653 wait complete. 2024-04-20T21:36:54.156 INFO:tasks.cephfs.filesystem:scrub status for tag:755ad1ed-c483-498c-98fd-b6fd781a7d3d - {'path': '/', 'tag': '755ad1ed-c483-498c-98fd-b6fd781a7d3d', 'options': 'recursive,force'} 2024-04-20T21:36:54.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:54 smithi007 ceph-mon[16867]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 130 MiB/s wr, 19.41k op/s 2024-04-20T21:36:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:54 smithi016 ceph-mon[21409]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 130 MiB/s wr, 19.41k op/s 2024-04-20T21:36:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:54 smithi186 ceph-mon[22157]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 130 MiB/s wr, 19.41k op/s 2024-04-20T21:36:55.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:55 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:36:55.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:55 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:55.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:55 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:36:55.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:55 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:36:55.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:55 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:55.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:55 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:36:55.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:55 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:36:55.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:55 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:36:55.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:55 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:36:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:56 smithi016 ceph-mon[21409]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 93 MiB/s wr, 13.98k op/s 2024-04-20T21:36:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:56 smithi186 ceph-mon[22157]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 93 MiB/s wr, 13.98k op/s 2024-04-20T21:36:56.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:56 smithi007 ceph-mon[16867]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 93 MiB/s wr, 13.98k op/s 2024-04-20T21:36:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:36:58 smithi016 ceph-mon[21409]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 82 MiB/s wr, 12.65k op/s 2024-04-20T21:36:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:36:58 smithi186 ceph-mon[22157]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 82 MiB/s wr, 12.65k op/s 2024-04-20T21:36:58.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:36:58 smithi007 ceph-mon[16867]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 82 MiB/s wr, 12.65k op/s 2024-04-20T21:37:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:00 smithi016 ceph-mon[21409]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 110 MiB/s wr, 17.66k op/s 2024-04-20T21:37:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:00 smithi186 ceph-mon[22157]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 110 MiB/s wr, 17.66k op/s 2024-04-20T21:37:00.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:00 smithi007 ceph-mon[16867]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 110 MiB/s wr, 17.66k op/s 2024-04-20T21:37:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:02 smithi016 ceph-mon[21409]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 113 MiB/s wr, 18.16k op/s 2024-04-20T21:37:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:02 smithi186 ceph-mon[22157]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 113 MiB/s wr, 18.16k op/s 2024-04-20T21:37:02.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:02 smithi007 ceph-mon[16867]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 113 MiB/s wr, 18.16k op/s 2024-04-20T21:37:04.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:04 smithi016 ceph-mon[21409]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 118 MiB/s wr, 18.04k op/s 2024-04-20T21:37:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:04 smithi186 ceph-mon[22157]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 118 MiB/s wr, 18.04k op/s 2024-04-20T21:37:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:04 smithi007 ceph-mon[16867]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 118 MiB/s wr, 18.04k op/s 2024-04-20T21:37:06.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:06 smithi016 ceph-mon[21409]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 96 MiB/s wr, 14.48k op/s 2024-04-20T21:37:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:06 smithi186 ceph-mon[22157]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 96 MiB/s wr, 14.48k op/s 2024-04-20T21:37:06.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:06 smithi007 ceph-mon[16867]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 96 MiB/s wr, 14.48k op/s 2024-04-20T21:37:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:37:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:37:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:37:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:37:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:37:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:37:07.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:37:07.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:08 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:08 smithi016 ceph-mon[21409]: pgmap v116: 97 pgs: 97 active+clean; 17 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 MiB/s wr, 15.97k op/s 2024-04-20T21:37:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:08 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:08 smithi186 ceph-mon[22157]: pgmap v116: 97 pgs: 97 active+clean; 17 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 MiB/s wr, 15.97k op/s 2024-04-20T21:37:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:08 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:37:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:08 smithi007 ceph-mon[16867]: pgmap v116: 97 pgs: 97 active+clean; 17 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 110 MiB/s wr, 15.97k op/s 2024-04-20T21:37:10.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:10 smithi016 ceph-mon[21409]: pgmap v117: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 135 MiB/s wr, 18.57k op/s 2024-04-20T21:37:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:10 smithi186 ceph-mon[22157]: pgmap v117: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 135 MiB/s wr, 18.57k op/s 2024-04-20T21:37:10.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:10 smithi007 ceph-mon[16867]: pgmap v117: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 135 MiB/s wr, 18.57k op/s 2024-04-20T21:37:12.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:12 smithi016 ceph-mon[21409]: pgmap v118: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 MiB/s wr, 13.99k op/s 2024-04-20T21:37:12.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:12 smithi186 ceph-mon[22157]: pgmap v118: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 MiB/s wr, 13.99k op/s 2024-04-20T21:37:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:12 smithi007 ceph-mon[16867]: pgmap v118: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 111 MiB/s wr, 13.99k op/s 2024-04-20T21:37:13.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:13 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:13.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:13 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:13.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:13 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:14 smithi007 ceph-mon[16867]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 121 MiB/s wr, 15.24k op/s 2024-04-20T21:37:14.845 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:14 smithi016 ceph-mon[21409]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 121 MiB/s wr, 15.24k op/s 2024-04-20T21:37:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:14 smithi186 ceph-mon[22157]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 121 MiB/s wr, 15.24k op/s 2024-04-20T21:37:16.790 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:16 smithi007 ceph-mon[16867]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 MiB/s wr, 12.53k op/s 2024-04-20T21:37:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:16 smithi016 ceph-mon[21409]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 MiB/s wr, 12.53k op/s 2024-04-20T21:37:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:16 smithi186 ceph-mon[22157]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 97 MiB/s wr, 12.53k op/s 2024-04-20T21:37:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:37:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:37:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:37:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:37:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:37:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:37:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:37:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:37:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:37:18.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:18 smithi007 ceph-mon[16867]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 101 MiB/s wr, 13.42k op/s 2024-04-20T21:37:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:18 smithi016 ceph-mon[21409]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 101 MiB/s wr, 13.42k op/s 2024-04-20T21:37:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:18 smithi186 ceph-mon[22157]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 101 MiB/s wr, 13.42k op/s 2024-04-20T21:37:19.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:19 smithi016 ceph-mon[21409]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-04-20T21:37:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:19 smithi186 ceph-mon[22157]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-04-20T21:37:20.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:19 smithi007 ceph-mon[16867]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 92 MiB/s wr, 13.33k op/s 2024-04-20T21:37:22.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:22 smithi186 ceph-mon[22157]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 MiB/s wr, 9.74k op/s 2024-04-20T21:37:22.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:22 smithi007 ceph-mon[16867]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 MiB/s wr, 9.74k op/s 2024-04-20T21:37:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:22 smithi016 ceph-mon[21409]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 MiB/s wr, 9.74k op/s 2024-04-20T21:37:24.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:37:24.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:37:24.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:24 smithi186 ceph-mon[22157]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 MiB/s wr, 14.14k op/s 2024-04-20T21:37:24.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 -- 172.21.15.7:0/554958020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 msgr2=0x7fbfd010e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/554958020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd010e9d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009ca0 tx=0x7fbfc802c1b0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 -- 172.21.15.7:0/554958020 shutdown_connections 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/554958020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd010e9d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/554958020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd0103af0 0x7fbfd010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/554958020 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfd01031d0 0x7fbfd01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 -- 172.21.15.7:0/554958020 >> 172.21.15.7:0/554958020 conn(0x7fbfd0077f50 msgr2=0x7fbfd0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 -- 172.21.15.7:0/554958020 shutdown_connections 2024-04-20T21:37:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.513+0000 7fbfd7a00640 1 -- 172.21.15.7:0/554958020 wait complete. 2024-04-20T21:37:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.515+0000 7fbfd7a00640 1 Processor -- start 2024-04-20T21:37:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.515+0000 7fbfd7a00640 1 -- start start 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 0x7fbfd019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfd0103af0 0x7fbfd01964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0110160 con 0x7fbfd01031d0 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010ffe0 con 0x7fbfd010c5a0 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.516+0000 7fbfd7a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01102e0 con 0x7fbfd0103af0 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46060/0 (socket says 172.21.15.7:46060) 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 -- 172.21.15.7:0/2041751169 learned_addr learned my addr 172.21.15.7:0/2041751169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5000640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 0x7fbfd019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 -- 172.21.15.7:0/2041751169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfd0103af0 msgr2=0x7fbfd01964a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfd0103af0 0x7fbfd01964a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.517+0000 7fbfd5a00640 1 -- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 msgr2=0x7fbfd019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.518+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 0x7fbfd019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.518+0000 7fbfd5a00640 1 -- 172.21.15.7:0/2041751169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd0196fa0 con 0x7fbfd010c5a0 2024-04-20T21:37:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.518+0000 7fbfd5000640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 0x7fbfd019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.519+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8037040 tx=0x7fbfc8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.520+0000 7fbfcd600640 1 -- 172.21.15.7:0/2041751169 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8039040 con 0x7fbfd010c5a0 2024-04-20T21:37:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.520+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd0197230 con 0x7fbfd010c5a0 2024-04-20T21:37:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.520+0000 7fbfcd600640 1 -- 172.21.15.7:0/2041751169 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8044070 con 0x7fbfd010c5a0 2024-04-20T21:37:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.520+0000 7fbfcd600640 1 -- 172.21.15.7:0/2041751169 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80329e0 con 0x7fbfd010c5a0 2024-04-20T21:37:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.520+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd019b230 con 0x7fbfd010c5a0 2024-04-20T21:37:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.530+0000 7fbfcd600640 1 -- 172.21.15.7:0/2041751169 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbfc8032b80 con 0x7fbfd010c5a0 2024-04-20T21:37:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.532+0000 7fbfda63b740 1 Processor -- start 2024-04-20T21:37:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.533+0000 7fbfda63b740 1 -- start start 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.535+0000 7fbfcd600640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 0x7fbfb80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfcd600640 1 -- 172.21.15.7:0/2041751169 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfc80f2aa0 con 0x7fbfd010c5a0 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd441a0 0x5594ecd44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594eccaa150 con 0x5594ecd44ac0 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594ecc50370 con 0x5594ecd441a0 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.536+0000 7fbfda63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5594ecca6110 con 0x5594ecd4d6a0 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41306/0 (socket says 172.21.15.7:41306) 2024-04-20T21:37:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfd5a00640 1 -- 172.21.15.7:0/75363482 learned_addr learned my addr 172.21.15.7:0/75363482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 0x7fbfb80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.538 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:24 smithi007 ceph-mon[16867]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 MiB/s wr, 14.14k op/s 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.537+0000 7fbfd5000640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd441a0 0x5594ecd44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfd5a00640 1 -- 172.21.15.7:0/75363482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd441a0 msgr2=0x5594ecd44580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd441a0 0x5594ecd44580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfd5a00640 1 -- 172.21.15.7:0/75363482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 msgr2=0x5594ecd4d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfd5a00640 1 -- 172.21.15.7:0/75363482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594eca867d0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.538+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:24.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.539+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c7280 tx=0x7fbfc80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a19b08950c4d7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.541+0000 7fbfc2000640 1 -- 172.21.15.7:0/75363482 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80bf2d0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.541+0000 7fbfc2000640 1 -- 172.21.15.7:0/75363482 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc80bebd0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.541+0000 7fbfc2000640 1 -- 172.21.15.7:0/75363482 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80f0460 con 0x5594ecd4d6a0 2024-04-20T21:37:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.542+0000 7fbfda63b740 1 -- 172.21.15.7:0/75363482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 msgr2=0x5594ecd4fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.542+0000 7fbfda63b740 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c7280 tx=0x7fbfc80095c0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.542+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 0x7fbfb80a62b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5594ecd43c70 tx=0x7fbfbc005cd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 -- 172.21.15.7:0/75363482 shutdown_connections 2024-04-20T21:37:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4fa90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 --2- 172.21.15.7:0/75363482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd441a0 0x5594ecd44580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 -- 172.21.15.7:0/75363482 >> 172.21.15.7:0/75363482 conn(0x5594ecd3e8d0 msgr2=0x5594ecd3ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 -- 172.21.15.7:0/75363482 shutdown_connections 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.544+0000 7fbfda63b740 1 -- 172.21.15.7:0/75363482 wait complete. 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.545+0000 7fbfda63b740 1 -- start start 2024-04-20T21:37:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.545+0000 7fbfda63b740 1 Processor -- start 2024-04-20T21:37:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.546+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd441a0 0x5594ecd4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.546+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.546+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfda63b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5594ecd50250 con 0x5594ecd44ac0 2024-04-20T21:37:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfda63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5594ecc63740 con 0x5594ecd4d6a0 2024-04-20T21:37:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfda63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5594ecd503d0 con 0x5594ecd441a0 2024-04-20T21:37:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfda63b740 10 client.0 _ll_register_callbacks cb 0x5594ecc5eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfd5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:46082/0 (socket says 172.21.15.7:46082) 2024-04-20T21:37:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfd5a00640 1 -- 172.21.15.7:0/1164572581 learned_addr learned my addr 172.21.15.7:0/1164572581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:24.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.547+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfd5a00640 1 -- 172.21.15.7:0/1164572581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd441a0 msgr2=0x5594ecd4a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd441a0 0x5594ecd4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfd5a00640 1 -- 172.21.15.7:0/1164572581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 msgr2=0x5594ecd4ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfd5a00640 1 -- 172.21.15.7:0/1164572581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ec9bc3f0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.548+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.549+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80beb60 tx=0x7fbfc80bb4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e9052600ff382a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.549+0000 7fbfb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.549+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80d8060 con 0x5594ecd4d6a0 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.550+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfc8046310 con 0x5594ecd4d6a0 2024-04-20T21:37:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.550+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc80bebd0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.550+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfc8046510 con 0x5594ecd4d6a0 2024-04-20T21:37:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.550+0000 7fbfda63b740 10 client.26084 fetch_fsmap learned FSMap version 26 2024-04-20T21:37:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.550+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594ec97c0f0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.552+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc80bebd0 con 0x5594ecd4d6a0 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.552+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8343+0+0 (secure 0 0 0) 0x7fbfc803c030 con 0x5594ecd4d6a0 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 10 client.26084 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 20 client.26084 populate_metadata read hostname 'smithi007' 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecd5d5f0 0x5594ecd5fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 4 client.26084 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.553+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594eccb7b70 con 0x5594ecd5d5f0 2024-04-20T21:37:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.554+0000 7fbfd5000640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecd5d5f0 0x5594ecd5fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.562+0000 7fbfd5000640 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecd5d5f0 0x5594ecd5fa00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abbe4462020cf6d server_cookie=33eb50246c08909b in_seq=0 out_seq=0 2024-04-20T21:37:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.562+0000 7fbfb6a00640 10 client.26084 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:37:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.563+0000 7fbfb6a00640 1 -- 172.21.15.7:0/1164572581 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594eccb7b70 con 0x5594ecd5d5f0 2024-04-20T21:37:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.563+0000 7fbfb6a00640 10 client.26084 handle_command_reply: tid=0 2024-04-20T21:37:24.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.563+0000 7fbfda63b740 1 client.26084 shutdown 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 msgr2=0x5594ecd4b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80beb60 tx=0x7fbfc80bb4e0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 shutdown_connections 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecd4b220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecd5d5f0 0x5594ecd5fa00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd44ac0 0x5594ecd4ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 --2- 172.21.15.7:0/1164572581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd441a0 0x5594ecd4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.564+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 >> 172.21.15.7:0/1164572581 conn(0x5594ecd3e8d0 msgr2=0x5594ecd40620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.565+0000 7fbfb6a00640 0 client.26084 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:37:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.565+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 shutdown_connections 2024-04-20T21:37:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.565+0000 7fbfda63b740 1 -- 172.21.15.7:0/1164572581 wait complete. 2024-04-20T21:37:24.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.565+0000 7fbfda63b740 20 client.26084 trim_cache size 0 max 16384 2024-04-20T21:37:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 Processor -- start 2024-04-20T21:37:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 -- start start 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 0x5594ecd496e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd59000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bf410 con 0x5594ecd441a0 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bccd0 con 0x5594ecd44ac0 2024-04-20T21:37:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.639+0000 7fbfda63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bab20 con 0x5594ecd4d6a0 2024-04-20T21:37:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.640+0000 7fbfd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.640+0000 7fbfcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 0x5594ecd496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.640+0000 7fbfd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52420/0 (socket says 172.21.15.7:52420) 2024-04-20T21:37:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.640+0000 7fbfd5000640 1 -- 172.21.15.7:0/3238362007 learned_addr learned my addr 172.21.15.7:0/3238362007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:24.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.640+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd59000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.641+0000 7fbfcfe00640 1 -- 172.21.15.7:0/3238362007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 msgr2=0x5594ecd59000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.642+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd59000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.642+0000 7fbfcfe00640 1 -- 172.21.15.7:0/3238362007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 msgr2=0x5594ecd4ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.642+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.642+0000 7fbfcfe00640 1 -- 172.21.15.7:0/3238362007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ecd448f0 con 0x5594ecd44ac0 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.642+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd59000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:24.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.643+0000 7fbfd5000640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:24.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.643+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 0x5594ecd496e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5594ecd4bd70 tx=0x7fbfbc011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1499a1ce926266d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.645+0000 7fbfc1600640 1 -- 172.21.15.7:0/3238362007 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc021070 con 0x5594ecd44ac0 2024-04-20T21:37:24.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.645+0000 7fbfc1600640 1 -- 172.21.15.7:0/3238362007 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc00ecd0 con 0x5594ecd44ac0 2024-04-20T21:37:24.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.645+0000 7fbfc1600640 1 -- 172.21.15.7:0/3238362007 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc015040 con 0x5594ecd44ac0 2024-04-20T21:37:24.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.646+0000 7fbfda63b740 1 -- 172.21.15.7:0/3238362007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 msgr2=0x5594ecd496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.646+0000 7fbfda63b740 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 0x5594ecd496e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x5594ecd4bd70 tx=0x7fbfbc011000 comp rx=0 tx=0).stop 2024-04-20T21:37:24.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 -- 172.21.15.7:0/3238362007 shutdown_connections 2024-04-20T21:37:24.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd44ac0 0x5594ecd496e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd4d6a0 0x5594ecd59000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 --2- 172.21.15.7:0/3238362007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 -- 172.21.15.7:0/3238362007 >> 172.21.15.7:0/3238362007 conn(0x5594ecd3e8d0 msgr2=0x5594ecd405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 -- 172.21.15.7:0/3238362007 shutdown_connections 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.647+0000 7fbfda63b740 1 -- 172.21.15.7:0/3238362007 wait complete. 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.648+0000 7fbfda63b740 1 -- start start 2024-04-20T21:37:24.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.648+0000 7fbfda63b740 1 Processor -- start 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecdc2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bb190 con 0x5594ecd441a0 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bc600 con 0x5594ecd4d6a0 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01bb860 con 0x5594ecd44ac0 2024-04-20T21:37:24.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.649+0000 7fbfda63b740 10 client.0 _ll_register_callbacks cb 0x7fbfc80be9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.650+0000 7fbfcfe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.650+0000 7fbfcfe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41314/0 (socket says 172.21.15.7:41314) 2024-04-20T21:37:24.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.650+0000 7fbfcfe00640 1 -- 172.21.15.7:0/2316810273 learned_addr learned my addr 172.21.15.7:0/2316810273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.651+0000 7fbfd5000640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.651+0000 7fbfd5a00640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecdc2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.653+0000 7fbfcfe00640 1 -- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 msgr2=0x5594ecdc2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.653+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecdc2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.653+0000 7fbfcfe00640 1 -- 172.21.15.7:0/2316810273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 msgr2=0x5594ecd512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.653+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.653+0000 7fbfcfe00640 1 -- 172.21.15.7:0/2316810273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ecd520f0 con 0x5594ecd44ac0 2024-04-20T21:37:24.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.654+0000 7fbfcfe00640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005a90 tx=0x7fbfbc00ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e581bf8bbc88fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.654+0000 7fbfb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:37:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.655+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc033060 con 0x5594ecd44ac0 2024-04-20T21:37:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.655+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfbc00ecd0 con 0x5594ecd44ac0 2024-04-20T21:37:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.655+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfbc01d880 con 0x5594ecd44ac0 2024-04-20T21:37:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.658+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfbc021070 con 0x5594ecd44ac0 2024-04-20T21:37:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.658+0000 7fbfda63b740 10 client.26059 fetch_fsmap learned FSMap version 26 2024-04-20T21:37:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.658+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5594ecdd5290 con 0x5594ecd44ac0 2024-04-20T21:37:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.659+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfbc00ecd0 con 0x5594ecd44ac0 2024-04-20T21:37:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.659+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 26) ==== 8343+0+0 (secure 0 0 0) 0x7fbfbc02d030 con 0x5594ecd44ac0 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 10 client.26059 fetch_fsmap finished waiting for FSMap version 26 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 10 client.26059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 20 client.26059 populate_metadata read hostname 'smithi007' 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecdd59a0 0x5594ecdd7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 4 client.26059 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594eccb19e0 con 0x5594ecdd59a0 2024-04-20T21:37:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.660+0000 7fbfd5000640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecdd59a0 0x5594ecdd7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.661+0000 7fbfd5000640 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecdd59a0 0x5594ecdd7dc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb879f769c059e9 server_cookie=5e599e6ba1fa7699 in_seq=0 out_seq=0 2024-04-20T21:37:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.661+0000 7fbfb6a00640 10 client.26059 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:37:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.661+0000 7fbfb6a00640 1 -- 172.21.15.7:0/2316810273 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5594eccb19e0 con 0x5594ecdd59a0 2024-04-20T21:37:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.661+0000 7fbfb6a00640 10 client.26059 handle_command_reply: tid=0 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.661+0000 7fbfda63b740 1 client.26059 shutdown 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 msgr2=0x5594ecd517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc005a90 tx=0x7fbfbc00ef30 comp rx=0 tx=0).stop 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 shutdown_connections 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5594ecd4d6a0 0x5594ecdc2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5594ecdd59a0 0x5594ecdd7dc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5594ecd44ac0 0x5594ecd517f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 --2- 172.21.15.7:0/2316810273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5594ecd441a0 0x5594ecd512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 >> 172.21.15.7:0/2316810273 conn(0x5594ecd3e8d0 msgr2=0x5594ecd3fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.662+0000 7fbfb6a00640 0 client.26059 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:37:24.662 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:37:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:37:24.662 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:37:24.662 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.663+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 shutdown_connections 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.663+0000 7fbfda63b740 1 -- 172.21.15.7:0/2316810273 wait complete. 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.663+0000 7fbfda63b740 20 client.26059 trim_cache size 0 max 16384 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 msgr2=0x7fbfb80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 0x7fbfb80a62b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5594ecd43c70 tx=0x7fbfbc005cd0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 msgr2=0x7fbfd01969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8037040 tx=0x7fbfc8003ba0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 shutdown_connections 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfb80a3df0 0x7fbfb80a62b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfd010c5a0 0x7fbfd01969e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbfd0103af0 0x7fbfd01964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 --2- 172.21.15.7:0/2041751169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfd01031d0 0x7fbfd019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 >> 172.21.15.7:0/2041751169 conn(0x7fbfd0077f50 msgr2=0x7fbfd00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 shutdown_connections 2024-04-20T21:37:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:24.664+0000 7fbfd7a00640 1 -- 172.21.15.7:0/2041751169 wait complete. 2024-04-20T21:37:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:24 smithi016 ceph-mon[21409]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 80 MiB/s wr, 14.14k op/s 2024-04-20T21:37:24.696 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:37:24.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 -- 172.21.15.7:0/3800135770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac109920 msgr2=0x7fdbac117460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3800135770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac109920 0x7fdbac117460 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009930 tx=0x7fdb9c02bc20 comp rx=0 tx=0).stop 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 -- 172.21.15.7:0/3800135770 shutdown_connections 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3800135770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac109920 0x7fdbac117460 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3800135770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3800135770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac10f640 0x7fdbac10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.051+0000 7fdbb4800640 1 -- 172.21.15.7:0/3800135770 >> 172.21.15.7:0/3800135770 conn(0x7fdbac06d320 msgr2=0x7fdbac06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.053+0000 7fdbb4800640 1 -- 172.21.15.7:0/3800135770 shutdown_connections 2024-04-20T21:37:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.053+0000 7fdbb4800640 1 -- 172.21.15.7:0/3800135770 wait complete. 2024-04-20T21:37:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.055+0000 7fdbb4800640 1 Processor -- start 2024-04-20T21:37:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.055+0000 7fdbb4800640 1 -- start start 2024-04-20T21:37:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.056+0000 7fdbb4800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.056+0000 7fdbb4800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 0x7fdbac1143d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.056+0000 7fdbb4800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 0x7fdbac114910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.056+0000 7fdbb1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb4800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac11cb80 con 0x7fdbac108f60 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb4800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac11ca00 con 0x7fdbac10f640 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb4800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac11cd00 con 0x7fdbac109920 2024-04-20T21:37:25.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52440/0 (socket says 172.21.15.7:52440) 2024-04-20T21:37:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb1e00640 1 -- 172.21.15.7:0/3358797872 learned_addr learned my addr 172.21.15.7:0/3358797872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb1400640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 0x7fdbac1143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:25.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb2800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 0x7fdbac114910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb2800640 1 -- 172.21.15.7:0/3358797872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 msgr2=0x7fdbac1143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb2800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 0x7fdbac1143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.057+0000 7fdbb2800640 1 -- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 msgr2=0x7fdbac1172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.058+0000 7fdbb2800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.058+0000 7fdbb2800640 1 -- 172.21.15.7:0/3358797872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac1151a0 con 0x7fdbac10f640 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.058+0000 7fdbb1400640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 0x7fdbac1143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:37:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.058+0000 7fdbb2800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 0x7fdbac114910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009850 tx=0x7fdb9c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.058+0000 7fdbb1e00640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.059+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c043020 con 0x7fdbac10f640 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.059+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c0054e0 con 0x7fdbac10f640 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.059+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c003b50 con 0x7fdbac10f640 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.060+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac1b3920 con 0x7fdbac10f640 2024-04-20T21:37:25.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.060+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1b3df0 con 0x7fdbac10f640 2024-04-20T21:37:25.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.062+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac073ba0 con 0x7fdbac10f640 2024-04-20T21:37:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.065+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdb9c046020 con 0x7fdbac10f640 2024-04-20T21:37:25.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.068+0000 7fdba2a00640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 0x7fdb980a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:25.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.068+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdb9c0f26d0 con 0x7fdbac10f640 2024-04-20T21:37:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.069+0000 7fdbb1e00640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 0x7fdb980a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.072+0000 7fdbb1e00640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 0x7fdb980a6220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba4009720 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.079+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fdb9c04d020 con 0x7fdbac10f640 2024-04-20T21:37:25.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.689+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbac119410 con 0x7fdbac10f640 2024-04-20T21:37:25.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.694+0000 7fdba2a00640 1 -- 172.21.15.7:0/3358797872 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11870 (secure 0 0 0) 0x7fdb9c0befd0 con 0x7fdbac10f640 2024-04-20T21:37:25.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:37:25.693 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:11.448170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24535,24559,24575]},"id":1}]} 2024-04-20T21:37:25.693 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-04-20T21:37:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.698+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 msgr2=0x7fdb980a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.698+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 0x7fdb980a6220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009fd0 tx=0x7fdba4009720 comp rx=0 tx=0).stop 2024-04-20T21:37:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.698+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 msgr2=0x7fdbac114910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.698+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 0x7fdbac114910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009850 tx=0x7fdb9c0045e0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 shutdown_connections 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb980a3d60 0x7fdb980a6220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbac10f640 0x7fdbac114910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbac109920 0x7fdbac1143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 --2- 172.21.15.7:0/3358797872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbac108f60 0x7fdbac1172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 >> 172.21.15.7:0/3358797872 conn(0x7fdbac06d320 msgr2=0x7fdbac071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 shutdown_connections 2024-04-20T21:37:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:25.699+0000 7fdbb4800640 1 -- 172.21.15.7:0/3358797872 wait complete. 2024-04-20T21:37:25.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:37:26.406 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:26 smithi007 ceph-mon[16867]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 11.51k op/s 2024-04-20T21:37:26.406 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:26 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3358797872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:26 smithi016 ceph-mon[21409]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 11.51k op/s 2024-04-20T21:37:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:26 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3358797872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:26 smithi186 ceph-mon[22157]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 61 MiB/s wr, 11.51k op/s 2024-04-20T21:37:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:26 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3358797872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:37:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:28 smithi016 ceph-mon[21409]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 11.30k op/s 2024-04-20T21:37:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:28 smithi016 ceph-mon[21409]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:28 smithi186 ceph-mon[22157]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 11.30k op/s 2024-04-20T21:37:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:28 smithi186 ceph-mon[22157]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:29.634 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:28 smithi007 ceph-mon[16867]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 MiB/s wr, 11.30k op/s 2024-04-20T21:37:29.634 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:28 smithi007 ceph-mon[16867]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-20T21:37:30.535 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:30 smithi007 ceph-mon[16867]: pgmap v127: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 MiB/s wr, 14.65k op/s 2024-04-20T21:37:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:30 smithi016 ceph-mon[21409]: pgmap v127: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 MiB/s wr, 14.65k op/s 2024-04-20T21:37:30.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:30 smithi186 ceph-mon[22157]: pgmap v127: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 74 MiB/s wr, 14.65k op/s 2024-04-20T21:37:32.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:32 smithi007 ceph-mon[16867]: pgmap v128: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 12.37k op/s 2024-04-20T21:37:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:32 smithi016 ceph-mon[21409]: pgmap v128: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 12.37k op/s 2024-04-20T21:37:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:32 smithi186 ceph-mon[22157]: pgmap v128: 97 pgs: 97 active+clean; 18 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 62 MiB/s wr, 12.37k op/s 2024-04-20T21:37:34.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:34 smithi007 ceph-mon[16867]: pgmap v129: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 78 MiB/s wr, 15.67k op/s 2024-04-20T21:37:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:34 smithi016 ceph-mon[21409]: pgmap v129: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 78 MiB/s wr, 15.67k op/s 2024-04-20T21:37:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:34 smithi186 ceph-mon[22157]: pgmap v129: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 78 MiB/s wr, 15.67k op/s 2024-04-20T21:37:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:36 smithi016 ceph-mon[21409]: pgmap v130: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 65 MiB/s wr, 12.81k op/s 2024-04-20T21:37:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:36 smithi186 ceph-mon[22157]: pgmap v130: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 65 MiB/s wr, 12.81k op/s 2024-04-20T21:37:36.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:36 smithi007 ceph-mon[16867]: pgmap v130: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 65 MiB/s wr, 12.81k op/s 2024-04-20T21:37:37.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:37 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:37.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:37 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:37.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:37 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:37:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:38 smithi016 ceph-mon[21409]: pgmap v131: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 61 MiB/s wr, 12.09k op/s 2024-04-20T21:37:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:38 smithi186 ceph-mon[22157]: pgmap v131: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 61 MiB/s wr, 12.09k op/s 2024-04-20T21:37:38.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:38 smithi007 ceph-mon[16867]: pgmap v131: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 61 MiB/s wr, 12.09k op/s 2024-04-20T21:37:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:40 smithi016 ceph-mon[21409]: pgmap v132: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 81 MiB/s wr, 15.97k op/s 2024-04-20T21:37:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:40 smithi186 ceph-mon[22157]: pgmap v132: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 81 MiB/s wr, 15.97k op/s 2024-04-20T21:37:40.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:40 smithi007 ceph-mon[16867]: pgmap v132: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 81 MiB/s wr, 15.97k op/s 2024-04-20T21:37:42.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:42 smithi007 ceph-mon[16867]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 MiB/s wr, 11.49k op/s 2024-04-20T21:37:42.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:42 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:42.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:42 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:42.534 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:42 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:37:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:42 smithi016 ceph-mon[21409]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 MiB/s wr, 11.49k op/s 2024-04-20T21:37:42.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:42 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:42.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:42 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:42.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:42 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:37:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:42 smithi186 ceph-mon[22157]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 MiB/s wr, 11.49k op/s 2024-04-20T21:37:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:42 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:37:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:42 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:37:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:42 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:37:43.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:43 smithi016 ceph-mon[21409]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 MiB/s wr, 12.03k op/s 2024-04-20T21:37:43.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:43 smithi016 ceph-mon[21409]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T21:37:43.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:43 smithi186 ceph-mon[22157]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 MiB/s wr, 12.03k op/s 2024-04-20T21:37:43.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:43 smithi186 ceph-mon[22157]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T21:37:43.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:43 smithi007 ceph-mon[16867]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 60 MiB/s wr, 12.03k op/s 2024-04-20T21:37:43.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:43 smithi007 ceph-mon[16867]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T21:37:44.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:44 smithi007 ceph-mon[16867]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 11.98k op/s 2024-04-20T21:37:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:44 smithi016 ceph-mon[21409]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 11.98k op/s 2024-04-20T21:37:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:44 smithi186 ceph-mon[22157]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 60 MiB/s wr, 11.98k op/s 2024-04-20T21:37:47.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:47 smithi016 ceph-mon[21409]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 10.16k op/s 2024-04-20T21:37:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:47 smithi186 ceph-mon[22157]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 10.16k op/s 2024-04-20T21:37:47.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:47 smithi007 ceph-mon[16867]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 MiB/s wr, 10.16k op/s 2024-04-20T21:37:49.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:49 smithi016 ceph-mon[21409]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 11.63k op/s 2024-04-20T21:37:49.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:49 smithi186 ceph-mon[22157]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 11.63k op/s 2024-04-20T21:37:49.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:49 smithi007 ceph-mon[16867]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 58 MiB/s wr, 11.63k op/s 2024-04-20T21:37:51.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:51 smithi016 ceph-mon[21409]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 MiB/s wr, 7.84k op/s 2024-04-20T21:37:51.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:51 smithi186 ceph-mon[22157]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 MiB/s wr, 7.84k op/s 2024-04-20T21:37:51.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:51 smithi007 ceph-mon[16867]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 38 MiB/s wr, 7.84k op/s 2024-04-20T21:37:53.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:53 smithi007 ceph-mon[16867]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.53k op/s 2024-04-20T21:37:53.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:53 smithi016 ceph-mon[21409]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.53k op/s 2024-04-20T21:37:53.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:53 smithi186 ceph-mon[22157]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 36 MiB/s wr, 7.53k op/s 2024-04-20T21:37:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.171+0000 7f8eb9800640 1 -- 172.21.15.7:0/2768673735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb4109920 msgr2=0x7f8eb4115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.171+0000 7f8eb9800640 1 --2- 172.21.15.7:0/2768673735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb4109920 0x7f8eb4115670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00074e0 tx=0x7f8ea002c8d0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.172+0000 7f8eb9800640 1 -- 172.21.15.7:0/2768673735 shutdown_connections 2024-04-20T21:37:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.172+0000 7f8eb9800640 1 --2- 172.21.15.7:0/2768673735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb4109920 0x7f8eb4115670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.172+0000 7f8eb9800640 1 --2- 172.21.15.7:0/2768673735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4108f60 0x7f8eb41093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.172+0000 7f8eb9800640 1 --2- 172.21.15.7:0/2768673735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb410e840 0x7f8eb410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.172+0000 7f8eb9800640 1 -- 172.21.15.7:0/2768673735 >> 172.21.15.7:0/2768673735 conn(0x7f8eb406d2c0 msgr2=0x7f8eb406d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.174+0000 7f8eb9800640 1 -- 172.21.15.7:0/2768673735 shutdown_connections 2024-04-20T21:37:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.174+0000 7f8eb9800640 1 -- 172.21.15.7:0/2768673735 wait complete. 2024-04-20T21:37:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.176+0000 7f8eb9800640 1 Processor -- start 2024-04-20T21:37:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 -- start start 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 0x7f8eb4115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb410e840 0x7f8eb4112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41184e0 con 0x7f8eb4108f60 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4118360 con 0x7f8eb410e840 2024-04-20T21:37:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.177+0000 7f8eb9800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4118660 con 0x7f8eb4109920 2024-04-20T21:37:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41336/0 (socket says 172.21.15.7:41336) 2024-04-20T21:37:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 -- 172.21.15.7:0/592582018 learned_addr learned my addr 172.21.15.7:0/592582018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb410e840 0x7f8eb4112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 -- 172.21.15.7:0/592582018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb410e840 msgr2=0x7f8eb4112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb410e840 0x7f8eb4112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.178+0000 7f8eb3400640 1 -- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 msgr2=0x7f8eb4115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.179+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 0x7f8eb4115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.179+0000 7f8eb3400640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 0x7f8eb4115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.179+0000 7f8eb3400640 1 -- 172.21.15.7:0/592582018 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb41134d0 con 0x7f8eb4109920 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.179+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 0x7f8eb4115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb3400640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d770 tx=0x7f8ea400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb0c00640 1 -- 172.21.15.7:0/592582018 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4017070 con 0x7f8eb4109920 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb0c00640 1 -- 172.21.15.7:0/592582018 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea4005500 con 0x7f8eb4109920 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb0c00640 1 -- 172.21.15.7:0/592582018 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4010460 con 0x7f8eb4109920 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb41b3890 con 0x7f8eb4109920 2024-04-20T21:37:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.180+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41b3d90 con 0x7f8eb4109920 2024-04-20T21:37:54.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.192+0000 7f8eb0c00640 1 -- 172.21.15.7:0/592582018 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8ea4009c30 con 0x7f8eb4109920 2024-04-20T21:37:54.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.199+0000 7f8eb0c00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 0x7f8e980a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.199+0000 7f8eb0c00640 1 -- 172.21.15.7:0/592582018 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8ea40d0340 con 0x7f8eb4109920 2024-04-20T21:37:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.220+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 0x7f8e980a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.238+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 0x7f8e980a62e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4073b00 tx=0x7f8eac008370 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.250+0000 7f8ebdbec740 1 Processor -- start 2024-04-20T21:37:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 -- start start 2024-04-20T21:37:54.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23f4670 0x5600d23f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5600d2350050 con 0x5600d23f4670 2024-04-20T21:37:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600d22f6490 con 0x5600d23eba90 2024-04-20T21:37:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.256+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5600d234c010 con 0x5600d23eb170 2024-04-20T21:37:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.257+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.257+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41340/0 (socket says 172.21.15.7:41340) 2024-04-20T21:37:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.257+0000 7f8eb3e00640 1 -- 172.21.15.7:0/1611737976 learned_addr learned my addr 172.21.15.7:0/1611737976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.257+0000 7f8eb3400640 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.262+0000 7f8eb3e00640 1 -- 172.21.15.7:0/1611737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 msgr2=0x5600d23f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.262+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.262+0000 7f8eb3e00640 1 -- 172.21.15.7:0/1611737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23f4670 msgr2=0x5600d23f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:54.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.262+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23f4670 0x5600d23f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.262+0000 7f8eb3e00640 1 -- 172.21.15.7:0/1611737976 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d22fe070 con 0x5600d23eb170 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.263+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00f5b0 tx=0x7f8eac010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a661ed5234df003 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.264+0000 7f8ea9600640 1 -- 172.21.15.7:0/1611737976 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00e900 con 0x5600d23eb170 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.267+0000 7f8ea9600640 1 -- 172.21.15.7:0/1611737976 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac042070 con 0x5600d23eb170 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.267+0000 7f8ea9600640 1 -- 172.21.15.7:0/1611737976 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00e900 con 0x5600d23eb170 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.267+0000 7f8ebdbec740 1 -- 172.21.15.7:0/1611737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 msgr2=0x5600d23eb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.267+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00f5b0 tx=0x7f8eac010000 comp rx=0 tx=0).stop 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.271+0000 7f8ebdbec740 1 -- 172.21.15.7:0/1611737976 shutdown_connections 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.271+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.271+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23f4670 0x5600d23f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.271+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/1611737976 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23eb550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.271+0000 7f8ebdbec740 1 -- 172.21.15.7:0/1611737976 >> 172.21.15.7:0/1611737976 conn(0x5600d23e58a0 msgr2=0x5600d23e5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.272+0000 7f8ebdbec740 1 -- 172.21.15.7:0/1611737976 shutdown_connections 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.272+0000 7f8ebdbec740 1 -- 172.21.15.7:0/1611737976 wait complete. 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.273+0000 7f8ebdbec740 1 -- start start 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.273+0000 7f8ebdbec740 1 Processor -- start 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.273+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.274+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23eba90 0x5600d23f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23f4670 0x5600d23f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5600d23f7460 con 0x5600d23eba90 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600d2309860 con 0x5600d23f4670 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5600d23f75e0 con 0x5600d23eb170 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8ebdbec740 10 client.0 _ll_register_callbacks cb 0x5600d22d7fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41354/0 (socket says 172.21.15.7:41354) 2024-04-20T21:37:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.275+0000 7f8eb3e00640 1 -- 172.21.15.7:0/2348053277 learned_addr learned my addr 172.21.15.7:0/2348053277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.276+0000 7f8eb3e00640 1 -- 172.21.15.7:0/2348053277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23f4670 msgr2=0x5600d23f20e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.276+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23f4670 0x5600d23f20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.276+0000 7f8eb3e00640 1 -- 172.21.15.7:0/2348053277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23eba90 msgr2=0x5600d23f1ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.276+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23eba90 0x5600d23f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.276+0000 7f8eb3e00640 1 -- 172.21.15.7:0/2348053277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d230d950 con 0x5600d23eb170 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.277+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ecd0 tx=0x7f8eac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b7d346da427781 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.277+0000 7f8e9e000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.279+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac056060 con 0x5600d23eb170 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.279+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8eac00e700 con 0x5600d23eb170 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.280+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac042070 con 0x5600d23eb170 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.281+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8eac00e900 con 0x5600d23eb170 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.281+0000 7f8ebdbec740 10 client.26077 fetch_fsmap learned FSMap version 27 2024-04-20T21:37:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.281+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600d22db820 con 0x5600d23eb170 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.283+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac042070 con 0x5600d23eb170 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.283+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8eac004480 con 0x5600d23eb170 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 10 client.26077 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 10 client.26077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 20 client.26077 populate_metadata read hostname 'smithi007' 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d23ff4c0 0x5600d2401930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 4 client.26077 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.287+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d235da70 con 0x5600d23ff4c0 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.288+0000 7f8eb3400640 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d23ff4c0 0x5600d2401930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.289+0000 7f8eb3400640 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d23ff4c0 0x5600d2401930 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3c024cd28634602 server_cookie=9111ab3ef9bf1d8f in_seq=0 out_seq=0 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.289+0000 7f8e9e000640 10 client.26077 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.290+0000 7f8e9e000640 1 -- 172.21.15.7:0/2348053277 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600d235da70 con 0x5600d23ff4c0 2024-04-20T21:37:54.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.290+0000 7f8e9e000640 10 client.26077 handle_command_reply: tid=0 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.290+0000 7f8ebdbec740 1 client.26077 shutdown 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 msgr2=0x5600d23f1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ecd0 tx=0x7f8eac00ed00 comp rx=0 tx=0).stop 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 shutdown_connections 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d23ff4c0 0x5600d2401930 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23f4670 0x5600d23f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d23eba90 0x5600d23f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8e9e000640 0 client.26077 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/2348053277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23eb170 0x5600d23f1660 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.292+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 >> 172.21.15.7:0/2348053277 conn(0x5600d23e58a0 msgr2=0x5600d23e75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.293+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 shutdown_connections 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.293+0000 7f8ebdbec740 1 -- 172.21.15.7:0/2348053277 wait complete. 2024-04-20T21:37:54.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.293+0000 7f8ebdbec740 20 client.26077 trim_cache size 0 max 16384 2024-04-20T21:37:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.381+0000 7f8ebdbec740 1 Processor -- start 2024-04-20T21:37:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.382+0000 7f8ebdbec740 1 -- start start 2024-04-20T21:37:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.382+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.382+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.382+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.385+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb40769d0 con 0x5600d2406f30 2024-04-20T21:37:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.385+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4074350 con 0x5600d23eba90 2024-04-20T21:37:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.385+0000 7f8ebdbec740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4113b20 con 0x5600d23f4670 2024-04-20T21:37:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.386+0000 7f8eb8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.386+0000 7f8eb8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46352/0 (socket says 172.21.15.7:46352) 2024-04-20T21:37:54.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.386+0000 7f8eb8e00640 1 -- 172.21.15.7:0/4197278867 learned_addr learned my addr 172.21.15.7:0/4197278867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.389+0000 7f8eb8e00640 1 -- 172.21.15.7:0/4197278867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 msgr2=0x5600d23f1f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:37:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.389+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23f1f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.389+0000 7f8eb8e00640 1 -- 172.21.15.7:0/4197278867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 msgr2=0x5600d23eff50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:37:54.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.403+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23eff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.403+0000 7f8eb8e00640 1 -- 172.21.15.7:0/4197278867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d23eb8c0 con 0x5600d2406f30 2024-04-20T21:37:54.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.405+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0035000 tx=0x7f8ea000a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e5d2e185621d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.406+0000 7f8ea8c00640 1 -- 172.21.15.7:0/4197278867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000a2a0 con 0x5600d2406f30 2024-04-20T21:37:54.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.407+0000 7f8ea8c00640 1 -- 172.21.15.7:0/4197278867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea0007a50 con 0x5600d2406f30 2024-04-20T21:37:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.407+0000 7f8ea8c00640 1 -- 172.21.15.7:0/4197278867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000a2a0 con 0x5600d2406f30 2024-04-20T21:37:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 -- 172.21.15.7:0/4197278867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 msgr2=0x5600d24073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0035000 tx=0x7f8ea000a180 comp rx=0 tx=0).stop 2024-04-20T21:37:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 -- 172.21.15.7:0/4197278867 shutdown_connections 2024-04-20T21:37:54.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d24073b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/4197278867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d23eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.410+0000 7f8ebdbec740 1 -- 172.21.15.7:0/4197278867 >> 172.21.15.7:0/4197278867 conn(0x5600d23e58a0 msgr2=0x5600d23e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.411+0000 7f8ebdbec740 1 -- 172.21.15.7:0/4197278867 shutdown_connections 2024-04-20T21:37:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.412+0000 7f8ebdbec740 1 -- 172.21.15.7:0/4197278867 wait complete. 2024-04-20T21:37:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.412+0000 7f8ebdbec740 1 -- start start 2024-04-20T21:37:54.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.412+0000 7f8ebdbec740 1 Processor -- start 2024-04-20T21:37:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.415+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.415+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.418+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.418+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44426/0 (socket says 172.21.15.7:44426) 2024-04-20T21:37:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.418+0000 7f8ebdbec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d23eea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.419+0000 7f8eb3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.419+0000 7f8eb3400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41368/0 (socket says 172.21.15.7:41368) 2024-04-20T21:37:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.420+0000 7f8eb3e00640 1 -- 172.21.15.7:0/709271915 learned_addr learned my addr 172.21.15.7:0/709271915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:37:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.423+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4124040 con 0x5600d2406f30 2024-04-20T21:37:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.423+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb41254b0 con 0x5600d23eba90 2024-04-20T21:37:54.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.423+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4124710 con 0x5600d23f4670 2024-04-20T21:37:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.423+0000 7f8ebdbec740 10 client.0 _ll_register_callbacks cb 0x7f8eac01c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:37:54.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.423+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d23eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.424+0000 7f8eb8e00640 1 -- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 msgr2=0x5600d23ee540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.424+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.424+0000 7f8eb8e00640 1 -- 172.21.15.7:0/709271915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 msgr2=0x5600d24068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:37:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.424+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.424+0000 7f8eb8e00640 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d20623f0 con 0x5600d2406f30 2024-04-20T21:37:54.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.425+0000 7f8eb8e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d23eea80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002ce10 tx=0x7f8ea002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39fe8cf405e3fec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:37:54.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.425+0000 7f8e9e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:37:54.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.428+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000a940 con 0x5600d2406f30 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.429+0000 7f8eb3400640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.429+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.431+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea0007a50 con 0x5600d2406f30 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.431+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8ea0037040 con 0x5600d2406f30 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.431+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ea00046c0 con 0x5600d2406f30 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.431+0000 7f8ebdbec740 10 client.15933 fetch_fsmap learned FSMap version 27 2024-04-20T21:37:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.431+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600d212c7d0 con 0x5600d2406f30 2024-04-20T21:37:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.432+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea0007a50 con 0x5600d2406f30 2024-04-20T21:37:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.432+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8ea00035f0 con 0x5600d2406f30 2024-04-20T21:37:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.433+0000 7f8ebdbec740 10 client.15933 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:37:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.433+0000 7f8ebdbec740 10 client.15933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:37:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.433+0000 7f8ebdbec740 20 client.15933 populate_metadata read hostname 'smithi007' 2024-04-20T21:37:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.433+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d24aee10 0x5600d24b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:37:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.437+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d24aee10 0x5600d24b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:37:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.437+0000 7f8ebdbec740 4 client.15933 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:37:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.437+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600d23578e0 con 0x5600d24aee10 2024-04-20T21:37:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.438+0000 7f8eb3e00640 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d24aee10 0x5600d24b1230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c73438574df9665 server_cookie=b69bcfe68472a6d in_seq=0 out_seq=0 2024-04-20T21:37:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.438+0000 7f8e9e000640 10 client.15933 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:37:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.438+0000 7f8e9e000640 1 -- 172.21.15.7:0/709271915 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600d23578e0 con 0x5600d24aee10 2024-04-20T21:37:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.438+0000 7f8e9e000640 10 client.15933 handle_command_reply: tid=0 2024-04-20T21:37:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.438+0000 7f8ebdbec740 1 client.15933 shutdown 2024-04-20T21:37:54.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.444+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 msgr2=0x5600d23eea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.444+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d23eea80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ea002ce10 tx=0x7f8ea002c8d0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 shutdown_connections 2024-04-20T21:37:54.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600d2406f30 0x5600d23eea80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600d23f4670 0x5600d23ee540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600d24aee10 0x5600d24b1230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 --2- 172.21.15.7:0/709271915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600d23eba90 0x5600d24068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.445+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 >> 172.21.15.7:0/709271915 conn(0x5600d23e58a0 msgr2=0x5600d23e68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.447+0000 7f8e9e000640 0 client.15933 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:37:54.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.447+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 shutdown_connections 2024-04-20T21:37:54.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.447+0000 7f8ebdbec740 1 -- 172.21.15.7:0/709271915 wait complete. 2024-04-20T21:37:54.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.447+0000 7f8ebdbec740 20 client.15933 trim_cache size 0 max 16384 2024-04-20T21:37:54.452 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.456+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 msgr2=0x7f8e980a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.456+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 0x7f8e980a62e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4073b00 tx=0x7f8eac008370 comp rx=0 tx=0).stop 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.456+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 msgr2=0x7f8eb4112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.456+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d770 tx=0x7f8ea400dc30 comp rx=0 tx=0).stop 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 shutdown_connections 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8e980a3e20 0x7f8e980a62e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8eb410e840 0x7f8eb4112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8eb4109920 0x7f8eb4112690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 --2- 172.21.15.7:0/592582018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb4108f60 0x7f8eb4115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:37:54.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 >> 172.21.15.7:0/592582018 conn(0x7f8eb406d2c0 msgr2=0x7f8eb4072390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:37:54.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 shutdown_connections 2024-04-20T21:37:54.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:37:54.460+0000 7f8eb9800640 1 -- 172.21.15.7:0/592582018 wait complete. 2024-04-20T21:37:55.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:55 smithi007 ceph-mon[16867]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 10.31k op/s 2024-04-20T21:37:55.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:55 smithi016 ceph-mon[21409]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 10.31k op/s 2024-04-20T21:37:55.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:55 smithi186 ceph-mon[22157]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 50 MiB/s wr, 10.31k op/s 2024-04-20T21:37:55.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:37:57.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:57 smithi016 ceph-mon[21409]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 6.69k op/s 2024-04-20T21:37:57.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:57 smithi186 ceph-mon[22157]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 6.69k op/s 2024-04-20T21:37:57.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:57 smithi007 ceph-mon[16867]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 32 MiB/s wr, 6.69k op/s 2024-04-20T21:37:59.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:37:59 smithi016 ceph-mon[21409]: pgmap v142: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 10.86k op/s 2024-04-20T21:37:59.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:37:59 smithi186 ceph-mon[22157]: pgmap v142: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 10.86k op/s 2024-04-20T21:37:59.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:37:59 smithi007 ceph-mon[16867]: pgmap v142: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 53 MiB/s wr, 10.86k op/s 2024-04-20T21:38:01.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:01 smithi016 ceph-mon[21409]: pgmap v143: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 54 MiB/s wr, 10.92k op/s 2024-04-20T21:38:01.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:01 smithi186 ceph-mon[22157]: pgmap v143: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 54 MiB/s wr, 10.92k op/s 2024-04-20T21:38:01.785 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:01 smithi007 ceph-mon[16867]: pgmap v143: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 54 MiB/s wr, 10.92k op/s 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.884+0000 7f35a5400640 1 -- 172.21.15.7:0/3913950467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980a4dd0 msgr2=0x7f35980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.884+0000 7f35a5400640 1 --2- 172.21.15.7:0/3913950467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980a4dd0 0x7f35980a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f359001a910 tx=0x7f359003cd00 comp rx=0 tx=0).stop 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 -- 172.21.15.7:0/3913950467 shutdown_connections 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 --2- 172.21.15.7:0/3913950467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 --2- 172.21.15.7:0/3913950467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35980b7a60 0x7f35980b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 --2- 172.21.15.7:0/3913950467 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980a4dd0 0x7f35980a51b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 -- 172.21.15.7:0/3913950467 >> 172.21.15.7:0/3913950467 conn(0x7f359801a630 msgr2=0x7f359801aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:01.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 -- 172.21.15.7:0/3913950467 shutdown_connections 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.885+0000 7f35a5400640 1 -- 172.21.15.7:0/3913950467 wait complete. 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.887+0000 7f35a5400640 1 Processor -- start 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 -- start start 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35980a4dd0 0x7f35980b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980b7a60 0x7f35980b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35980af6e0 con 0x7f35980a4dd0 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35980af560 con 0x7f35980a56f0 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.888+0000 7f35a5400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f35980af860 con 0x7f35980b7a60 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.890+0000 7f359de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.890+0000 7f359de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44444/0 (socket says 172.21.15.7:44444) 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.890+0000 7f359de00640 1 -- 172.21.15.7:0/3019619279 learned_addr learned my addr 172.21.15.7:0/3019619279 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.890+0000 7f359f200640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980b7a60 0x7f35980b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 -- 172.21.15.7:0/3019619279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980b7a60 msgr2=0x7f35980b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980b7a60 0x7f35980b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 -- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35980a4dd0 msgr2=0x7f35980b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35980a4dd0 0x7f35980b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 -- 172.21.15.7:0/3019619279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980b3b00 con 0x7f35980a56f0 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.891+0000 7f359de00640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c920 tx=0x7f358c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.892+0000 7f3597400640 1 -- 172.21.15.7:0/3019619279 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c007c70 con 0x7f35980a56f0 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.892+0000 7f3597400640 1 -- 172.21.15.7:0/3019619279 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358c007e10 con 0x7f35980a56f0 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.892+0000 7f3597400640 1 -- 172.21.15.7:0/3019619279 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0189a0 con 0x7f35980a56f0 2024-04-20T21:38:01.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.894+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35980b3df0 con 0x7f35980a56f0 2024-04-20T21:38:01.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.894+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35980c3010 con 0x7f35980a56f0 2024-04-20T21:38:01.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.900+0000 7f3597400640 1 -- 172.21.15.7:0/3019619279 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f358c018b40 con 0x7f35980a56f0 2024-04-20T21:38:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.908+0000 7f35a7ff8740 1 Processor -- start 2024-04-20T21:38:01.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.908+0000 7f35a7ff8740 1 -- start start 2024-04-20T21:38:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.912+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.912+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e07130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.912+0000 7f359f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.912+0000 7f359f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41386/0 (socket says 172.21.15.7:41386) 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.912+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6e07670 0x55e2a6e09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.913+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6d63050 con 0x55e2a6e07670 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.913+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6d09490 con 0x55e2a6dfea90 2024-04-20T21:38:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.913+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6d5f010 con 0x55e2a6dfe170 2024-04-20T21:38:01.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.913+0000 7f359f200640 1 -- 172.21.15.7:0/2553373868 learned_addr learned my addr 172.21.15.7:0/2553373868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:01.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.913+0000 7f359de00640 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6e07670 0x55e2a6e09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.914+0000 7f359f200640 1 -- 172.21.15.7:0/2553373868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 msgr2=0x55e2a6e07130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:38:01.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.914+0000 7f359f200640 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e07130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.914+0000 7f359f200640 1 -- 172.21.15.7:0/2553373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6e07670 msgr2=0x55e2a6e09a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.914+0000 7f359f200640 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6e07670 0x55e2a6e09a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.914+0000 7f359f200640 1 -- 172.21.15.7:0/2553373868 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a6d11070 con 0x55e2a6dfe170 2024-04-20T21:38:01.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.915+0000 7f359f200640 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35a00676c0 tx=0x7f35a0069d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34ba0e49f318b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.915+0000 7f358b400640 1 -- 172.21.15.7:0/2553373868 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0074e40 con 0x55e2a6dfe170 2024-04-20T21:38:01.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.916+0000 7f358b400640 1 -- 172.21.15.7:0/2553373868 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0079070 con 0x55e2a6dfe170 2024-04-20T21:38:01.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.916+0000 7f358b400640 1 -- 172.21.15.7:0/2553373868 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a007d770 con 0x55e2a6dfe170 2024-04-20T21:38:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2553373868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 msgr2=0x55e2a6dfe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35a00676c0 tx=0x7f35a0069d20 comp rx=0 tx=0).stop 2024-04-20T21:38:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2553373868 shutdown_connections 2024-04-20T21:38:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e07130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6e07670 0x55e2a6e09a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2553373868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6dfe550 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.917+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2553373868 >> 172.21.15.7:0/2553373868 conn(0x55e2a6df88a0 msgr2=0x55e2a6df8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.925+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2553373868 shutdown_connections 2024-04-20T21:38:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.925+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2553373868 wait complete. 2024-04-20T21:38:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.925+0000 7f35a7ff8740 1 -- start start 2024-04-20T21:38:01.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.926+0000 7f35a7ff8740 1 Processor -- start 2024-04-20T21:38:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.927+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.927+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6e07670 0x55e2a6e05070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6e0a460 con 0x55e2a6dfe170 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f359e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f359e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44454/0 (socket says 172.21.15.7:44454) 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f359e800640 1 -- 172.21.15.7:0/4134509437 learned_addr learned my addr 172.21.15.7:0/4134509437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6d1c860 con 0x55e2a6dfea90 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e2a6e0a5e0 con 0x55e2a6e07670 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.928+0000 7f35a7ff8740 10 client.0 _ll_register_callbacks cb 0x55e2a6ceafe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.929+0000 7f359de00640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6e07670 0x55e2a6e05070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.930+0000 7f359f200640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.930+0000 7f359e800640 1 -- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6e07670 msgr2=0x55e2a6e05070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.930+0000 7f359e800640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6e07670 0x55e2a6e05070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.930+0000 7f359e800640 1 -- 172.21.15.7:0/4134509437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 msgr2=0x55e2a6e045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.930+0000 7f359e800640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.931+0000 7f359e800640 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a6d20950 con 0x55e2a6dfea90 2024-04-20T21:38:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.931+0000 7f359f200640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.931+0000 7f359e800640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f359003d240 tx=0x7f35900048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4a1bbd39a18024 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.931+0000 7f357fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.932+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590011020 con 0x55e2a6dfea90 2024-04-20T21:38:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.932+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a0079070 con 0x55e2a6dfea90 2024-04-20T21:38:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.932+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f359003dcc0 con 0x55e2a6dfea90 2024-04-20T21:38:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.933+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35900035f0 con 0x55e2a6dfea90 2024-04-20T21:38:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.933+0000 7f35a7ff8740 10 client.26111 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.933+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e2a6cee820 con 0x55e2a6dfea90 2024-04-20T21:38:01.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.933+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0079070 con 0x55e2a6dfea90 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f359000d760 con 0x55e2a6dfea90 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f35a7ff8740 10 client.26111 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f35a7ff8740 10 client.26111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f35a7ff8740 20 client.26111 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e175d0 0x55e2a6e19a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.934+0000 7f35a7ff8740 4 client.26111 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.935+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a6d70a70 con 0x55e2a6e175d0 2024-04-20T21:38:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.935+0000 7f359f200640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e175d0 0x55e2a6e19a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.937+0000 7f359f200640 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e175d0 0x55e2a6e19a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b25317cc114572a server_cookie=3bb2f5c6d580a10f in_seq=0 out_seq=0 2024-04-20T21:38:01.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.937+0000 7f357fe00640 10 client.26111 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.938+0000 7f357fe00640 1 -- 172.21.15.7:0/4134509437 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2a6d70a70 con 0x55e2a6e175d0 2024-04-20T21:38:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.938+0000 7f357fe00640 10 client.26111 handle_command_reply: tid=0 2024-04-20T21:38:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.939+0000 7f35a7ff8740 1 client.26111 shutdown 2024-04-20T21:38:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.939+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 msgr2=0x55e2a6e04b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.939+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f359003d240 tx=0x7f35900048f0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.939+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 shutdown_connections 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e175d0 0x55e2a6e19a50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6dfea90 0x55e2a6e04b30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6e07670 0x55e2a6e05070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/4134509437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 >> 172.21.15.7:0/4134509437 conn(0x55e2a6df88a0 msgr2=0x55e2a6dfa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f357fe00640 0 client.26111 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 shutdown_connections 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 1 -- 172.21.15.7:0/4134509437 wait complete. 2024-04-20T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.940+0000 7f35a7ff8740 20 client.26111 trim_cache size 0 max 16384 2024-04-20T21:38:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.948+0000 7f3597400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 0x7f35840a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.949+0000 7f3597400640 1 -- 172.21.15.7:0/3019619279 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f358c0d07a0 con 0x7f35980a56f0 2024-04-20T21:38:01.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.950+0000 7f359e800640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 0x7f35840a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.953+0000 7f359e800640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 0x7f35840a61a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3590018040 tx=0x7f3590005fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.972+0000 7f35a7ff8740 1 Processor -- start 2024-04-20T21:38:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.972+0000 7f35a7ff8740 1 -- start start 2024-04-20T21:38:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e06d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 0x55e2a6e13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e02ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35980ce320 con 0x55e2a6dfe170 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35980cbc60 con 0x55e2a6e07670 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f35980b45b0 con 0x55e2a6dfea90 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f359e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e06d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f359e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e06d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46410/0 (socket says 172.21.15.7:46410) 2024-04-20T21:38:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f359e800640 1 -- 172.21.15.7:0/443952291 learned_addr learned my addr 172.21.15.7:0/443952291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.973+0000 7f359de00640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 0x55e2a6e13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.974+0000 7f359f200640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e02ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359f200640 1 -- 172.21.15.7:0/443952291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 msgr2=0x55e2a6e13380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359f200640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 0x55e2a6e13380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359f200640 1 -- 172.21.15.7:0/443952291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 msgr2=0x55e2a6e06d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359f200640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e06d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359f200640 1 -- 172.21.15.7:0/443952291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a6e035a0 con 0x55e2a6e07670 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.976+0000 7f359de00640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 0x55e2a6e13380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:01.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.977+0000 7f359f200640 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e02ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35a0067fe0 tx=0x7f35a00678e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7bcbb36b05f1327 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.977+0000 7f358aa00640 1 -- 172.21.15.7:0/443952291 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0069240 con 0x55e2a6e07670 2024-04-20T21:38:01.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.978+0000 7f358aa00640 1 -- 172.21.15.7:0/443952291 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a0079040 con 0x55e2a6e07670 2024-04-20T21:38:01.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.978+0000 7f358aa00640 1 -- 172.21.15.7:0/443952291 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a008b020 con 0x55e2a6e07670 2024-04-20T21:38:01.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 -- 172.21.15.7:0/443952291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 msgr2=0x55e2a6e02ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e02ec0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35a0067fe0 tx=0x7f35a00678e0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 -- 172.21.15.7:0/443952291 shutdown_connections 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e02ec0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfea90 0x55e2a6e13380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/443952291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfe170 0x55e2a6e06d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.979+0000 7f35a7ff8740 1 -- 172.21.15.7:0/443952291 >> 172.21.15.7:0/443952291 conn(0x55e2a6df88a0 msgr2=0x55e2a6e04e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:01.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.980+0000 7f35a7ff8740 1 -- 172.21.15.7:0/443952291 shutdown_connections 2024-04-20T21:38:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.980+0000 7f35a7ff8740 1 -- 172.21.15.7:0/443952291 wait complete. 2024-04-20T21:38:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.980+0000 7f35a7ff8740 1 -- start start 2024-04-20T21:38:01.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 Processor -- start 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6e0b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35980ca120 con 0x55e2a6dfea90 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35980cb590 con 0x55e2a6e07670 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.981+0000 7f35a7ff8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f35980ca7f0 con 0x55e2a6dfe170 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.982+0000 7f35a7ff8740 10 client.0 _ll_register_callbacks cb 0x7f35900044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.982+0000 7f359de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.982+0000 7f359de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-04-20T21:38:01.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.982+0000 7f359de00640 1 -- 172.21.15.7:0/2795602033 learned_addr learned my addr 172.21.15.7:0/2795602033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.982+0000 7f359f200640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.983+0000 7f359e800640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6e0b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.985+0000 7f359de00640 1 -- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 msgr2=0x55e2a6e0b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.985+0000 7f359de00640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6e0b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.985+0000 7f359de00640 1 -- 172.21.15.7:0/2795602033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 msgr2=0x55e2a6e7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.985+0000 7f359de00640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.985+0000 7f359de00640 1 -- 172.21.15.7:0/2795602033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a6a753f0 con 0x55e2a6dfea90 2024-04-20T21:38:01.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.988+0000 7f359de00640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f358c09c6c0 tx=0x7f358c09c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7908c7817aa2b454 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:01.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.988+0000 7f357fe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.988+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c0da040 con 0x55e2a6dfea90 2024-04-20T21:38:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.989+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f358c0b2060 con 0x55e2a6dfea90 2024-04-20T21:38:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.990+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a0079040 con 0x55e2a6dfea90 2024-04-20T21:38:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.994+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f358c0da040 con 0x55e2a6dfea90 2024-04-20T21:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.995+0000 7f35a7ff8740 10 client.15939 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.995+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e2a6b3f7d0 con 0x55e2a6dfea90 2024-04-20T21:38:01.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.997+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f358c09ce30 con 0x55e2a6dfea90 2024-04-20T21:38:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.997+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f358c020020 con 0x55e2a6dfea90 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 10 client.15939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 20 client.15939 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e7e570 0x55e2a6e80990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 4 client.15939 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:01.999+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f35a0079040 con 0x55e2a6e7e570 2024-04-20T21:38:01.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.000+0000 7f359e800640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e7e570 0x55e2a6e80990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.002+0000 7f359e800640 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e7e570 0x55e2a6e80990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98e67b25a3a6735e server_cookie=89a90a36cb193000 in_seq=0 out_seq=0 2024-04-20T21:38:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.005+0000 7f357fe00640 10 client.15939 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.005+0000 7f357fe00640 1 -- 172.21.15.7:0/2795602033 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f35a0079040 con 0x55e2a6e7e570 2024-04-20T21:38:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.005+0000 7f357fe00640 10 client.15939 handle_command_reply: tid=0 2024-04-20T21:38:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.005+0000 7f35a7ff8740 1 client.15939 shutdown 2024-04-20T21:38:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.006+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 msgr2=0x55e2a6e0baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.006+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f358c09c6c0 tx=0x7f358c09c270 comp rx=0 tx=0).stop 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 shutdown_connections 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55e2a6e7e570 0x55e2a6e80990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e2a6e07670 0x55e2a6e7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e2a6dfea90 0x55e2a6e0baf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 --2- 172.21.15.7:0/2795602033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e2a6dfe170 0x55e2a6e0b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 >> 172.21.15.7:0/2795602033 conn(0x55e2a6df88a0 msgr2=0x55e2a6e04e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.007+0000 7f357fe00640 0 client.15939 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.008+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 shutdown_connections 2024-04-20T21:38:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.008+0000 7f35a7ff8740 1 -- 172.21.15.7:0/2795602033 wait complete. 2024-04-20T21:38:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.008+0000 7f35a7ff8740 20 client.15939 trim_cache size 0 max 16384 2024-04-20T21:38:02.009 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 msgr2=0x7f35840a61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 0x7f35840a61a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3590018040 tx=0x7f3590005fd0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 msgr2=0x7f35980b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c920 tx=0x7f358c00cde0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 shutdown_connections 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f35840a3ce0 0x7f35840a61a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35980a56f0 0x7f35980b3080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f35980b7a60 0x7f35980b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 --2- 172.21.15.7:0/3019619279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35980a4dd0 0x7f35980b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.011+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 >> 172.21.15.7:0/3019619279 conn(0x7f359801a630 msgr2=0x7f35980a3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.012+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 shutdown_connections 2024-04-20T21:38:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.012+0000 7f35a5400640 1 -- 172.21.15.7:0/3019619279 wait complete. 2024-04-20T21:38:02.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:38:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 -- 172.21.15.7:0/74707843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8109530 msgr2=0x7f65c8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 --2- 172.21.15.7:0/74707843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8109530 0x7f65c8109990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009900 tx=0x7f65c002bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 -- 172.21.15.7:0/74707843 shutdown_connections 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 --2- 172.21.15.7:0/74707843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8109530 0x7f65c8109990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 --2- 172.21.15.7:0/74707843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8114c00 0x7f65c8117010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 --2- 172.21.15.7:0/74707843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 -- 172.21.15.7:0/74707843 >> 172.21.15.7:0/74707843 conn(0x7f65c806ce70 msgr2=0x7f65c806d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 -- 172.21.15.7:0/74707843 shutdown_connections 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.380+0000 7f65c7200640 1 -- 172.21.15.7:0/74707843 wait complete. 2024-04-20T21:38:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.382+0000 7f65c7200640 1 Processor -- start 2024-04-20T21:38:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.382+0000 7f65c7200640 1 -- start start 2024-04-20T21:38:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.382+0000 7f65c7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c81a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 0x7f65c81a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65c811a940 con 0x7f65c806e240 2024-04-20T21:38:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c811a7c0 con 0x7f65c8114c00 2024-04-20T21:38:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c7200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65c811aac0 con 0x7f65c8109530 2024-04-20T21:38:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41418/0 (socket says 172.21.15.7:41418) 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c5400640 1 -- 172.21.15.7:0/715599384 learned_addr learned my addr 172.21.15.7:0/715599384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c81a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.383+0000 7f65c6800640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 0x7f65c81a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 -- 172.21.15.7:0/715599384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 msgr2=0x7f65c81a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 0x7f65c81a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 -- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 msgr2=0x7f65c81a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c6800640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 0x7f65c81a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c81a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 -- 172.21.15.7:0/715599384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c81a5eb0 con 0x7f65c8109530 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5e00640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c81a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.384+0000 7f65c5400640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f65c002c240 tx=0x7f65c00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.386+0000 7f65bea00640 1 -- 172.21.15.7:0/715599384 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0043020 con 0x7f65c8109530 2024-04-20T21:38:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.386+0000 7f65bea00640 1 -- 172.21.15.7:0/715599384 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00054e0 con 0x7f65c8109530 2024-04-20T21:38:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.386+0000 7f65bea00640 1 -- 172.21.15.7:0/715599384 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0003c50 con 0x7f65c8109530 2024-04-20T21:38:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.386+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c81a6140 con 0x7f65c8109530 2024-04-20T21:38:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.386+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c810a900 con 0x7f65c8109530 2024-04-20T21:38:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.391+0000 7f65cfbe3740 1 Processor -- start 2024-04-20T21:38:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.391+0000 7f65cfbe3740 1 -- start start 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fbf2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 0x563e1fc03940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e1fb57050 con 0x563e1fbf1f80 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e1fafd490 con 0x563e1fc03e80 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563e1fb53010 con 0x563e1fbf2930 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65c5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65c5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:44504/0 (socket says 172.21.15.7:44504) 2024-04-20T21:38:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65c5400640 1 -- 172.21.15.7:0/3562826712 learned_addr learned my addr 172.21.15.7:0/3562826712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.392+0000 7f65c6800640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fbf2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 0x563e1fc03940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 -- 172.21.15.7:0/3562826712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 msgr2=0x563e1fc03940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 0x563e1fc03940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 -- 172.21.15.7:0/3562826712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 msgr2=0x563e1fbf2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fbf2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 -- 172.21.15.7:0/3562826712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1fb05070 con 0x563e1fc03e80 2024-04-20T21:38:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 0x563e1fc03940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.393+0000 7f65c5400640 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006010 tx=0x7f65c0009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e263c6f68005a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.397+0000 7f65bea00640 1 -- 172.21.15.7:0/715599384 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f65c0046020 con 0x7f65c8109530 2024-04-20T21:38:02.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.397+0000 7f65b7400640 1 -- 172.21.15.7:0/3562826712 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00f4b30 con 0x563e1fc03e80 2024-04-20T21:38:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.397+0000 7f65b7400640 1 -- 172.21.15.7:0/3562826712 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00f53a0 con 0x563e1fc03e80 2024-04-20T21:38:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.397+0000 7f65b7400640 1 -- 172.21.15.7:0/3562826712 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00f6680 con 0x563e1fc03e80 2024-04-20T21:38:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.398+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3562826712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 msgr2=0x563e1fc06270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.398+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65c0006010 tx=0x7f65c0009560 comp rx=0 tx=0).stop 2024-04-20T21:38:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.399+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3562826712 shutdown_connections 2024-04-20T21:38:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.399+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fc03e80 0x563e1fc06270 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.399+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf2930 0x563e1fc03940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.399+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3562826712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fbf2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.399+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3562826712 >> 172.21.15.7:0/3562826712 conn(0x563e1fbec8a0 msgr2=0x563e1fbecc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.400+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3562826712 shutdown_connections 2024-04-20T21:38:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.400+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3562826712 wait complete. 2024-04-20T21:38:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.400+0000 7f65bea00640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 0x7f65ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.401+0000 7f65cfbe3740 1 -- start start 2024-04-20T21:38:02.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.401+0000 7f65cfbe3740 1 Processor -- start 2024-04-20T21:38:02.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.401+0000 7f65bea00640 1 -- 172.21.15.7:0/715599384 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f65c00f4560 con 0x7f65c8109530 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fc00f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc01a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563e1fc06a00 con 0x563e1fbf1f80 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563e1fb10860 con 0x563e1fbf2930 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65c6800640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 0x7f65ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563e1fc06b80 con 0x563e1fc03e80 2024-04-20T21:38:02.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65cfbe3740 10 client.0 _ll_register_callbacks cb 0x563e1fadefe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:02.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65c6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc01a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65c6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc01a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41440/0 (socket says 172.21.15.7:41440) 2024-04-20T21:38:02.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.404+0000 7f65c6800640 1 -- 172.21.15.7:0/765778932 learned_addr learned my addr 172.21.15.7:0/765778932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.405+0000 7f65c5e00640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fc00f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.405+0000 7f65c5400640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 -- 172.21.15.7:0/765778932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 msgr2=0x563e1fc01a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc01a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 -- 172.21.15.7:0/765778932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 msgr2=0x563e1fc00f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fc00f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 -- 172.21.15.7:0/765778932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1fb14950 con 0x563e1fbf2930 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.406+0000 7f65c5400640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc014c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x563e1fbedcc0 tx=0x7f65c00c05d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f914ab07960d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.407+0000 7f65abe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.407+0000 7f65c6800640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 0x7f65ac0a6270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65b8015c30 tx=0x7f65b80157b0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.407+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c00c0160 con 0x563e1fbf2930 2024-04-20T21:38:02.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.407+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65c00f53a0 con 0x563e1fbf2930 2024-04-20T21:38:02.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.408+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f65c00beb00 con 0x563e1fbf2930 2024-04-20T21:38:02.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.408+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65c00c0f00 con 0x563e1fbf2930 2024-04-20T21:38:02.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.408+0000 7f65cfbe3740 10 client.26129 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:02.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.408+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e1fae2820 con 0x563e1fbf2930 2024-04-20T21:38:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65c00f53a0 con 0x563e1fbf2930 2024-04-20T21:38:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f65c00c0a60 con 0x563e1fbf2930 2024-04-20T21:38:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 10 client.26129 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:02.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 20 client.26129 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc13a10 0x563e1fc15e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 4 client.26129 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.409+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1fb64a70 con 0x563e1fc13a10 2024-04-20T21:38:02.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.410+0000 7f65c5e00640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc13a10 0x563e1fc15e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.411+0000 7f65c5e00640 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc13a10 0x563e1fc15e00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8452aa648d1901d3 server_cookie=37155797d7fed5bd in_seq=0 out_seq=0 2024-04-20T21:38:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.411+0000 7f65abe00640 10 client.26129 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:02.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.412+0000 7f65abe00640 1 -- 172.21.15.7:0/765778932 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e1fb64a70 con 0x563e1fc13a10 2024-04-20T21:38:02.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.412+0000 7f65abe00640 10 client.26129 handle_command_reply: tid=0 2024-04-20T21:38:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.412+0000 7f65cfbe3740 1 client.26129 shutdown 2024-04-20T21:38:02.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 msgr2=0x563e1fc014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc014c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x563e1fbedcc0 tx=0x7f65c00c05d0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 shutdown_connections 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc014c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc13a10 0x563e1fc15e00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc01a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/765778932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf1f80 0x563e1fc00f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65abe00640 0 client.26129 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.414+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 >> 172.21.15.7:0/765778932 conn(0x563e1fbec8a0 msgr2=0x563e1fbee5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.415+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 shutdown_connections 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.415+0000 7f65cfbe3740 1 -- 172.21.15.7:0/765778932 wait complete. 2024-04-20T21:38:02.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.415+0000 7f65cfbe3740 20 client.26129 trim_cache size 0 max 16384 2024-04-20T21:38:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.446+0000 7f65cfbe3740 1 Processor -- start 2024-04-20T21:38:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.446+0000 7f65cfbe3740 1 -- start start 2024-04-20T21:38:02.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 0x563e1fc0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc01650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 0x563e1fc1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b6880 con 0x563e1fc03e80 2024-04-20T21:38:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b4140 con 0x563e1fbf2930 2024-04-20T21:38:02.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b1f90 con 0x563e1fbf1f80 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65c6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 0x563e1fc1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65c6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 0x563e1fc1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46462/0 (socket says 172.21.15.7:46462) 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.447+0000 7f65c6800640 1 -- 172.21.15.7:0/3504192437 learned_addr learned my addr 172.21.15.7:0/3504192437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 0x563e1fc0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5400640 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc01650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5e00640 1 -- 172.21.15.7:0/3504192437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 msgr2=0x563e1fc01650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc01650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5e00640 1 -- 172.21.15.7:0/3504192437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 msgr2=0x563e1fc1b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.448+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 0x563e1fc1b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.449+0000 7f65c5e00640 1 -- 172.21.15.7:0/3504192437 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1f8693f0 con 0x563e1fbf1f80 2024-04-20T21:38:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.451+0000 7f65c5e00640 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 0x563e1fc0f810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x563e1fc16540 tx=0x7f65b0012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=411adc9010a2bb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.452+0000 7f65b6a00640 1 -- 172.21.15.7:0/3504192437 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b001c050 con 0x563e1fbf1f80 2024-04-20T21:38:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.452+0000 7f65b6a00640 1 -- 172.21.15.7:0/3504192437 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0020030 con 0x563e1fbf1f80 2024-04-20T21:38:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.452+0000 7f65b6a00640 1 -- 172.21.15.7:0/3504192437 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0010990 con 0x563e1fbf1f80 2024-04-20T21:38:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.453+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3504192437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 msgr2=0x563e1fc0f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.453+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 0x563e1fc0f810 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x563e1fc16540 tx=0x7f65b0012820 comp rx=0 tx=0).stop 2024-04-20T21:38:02.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3504192437 shutdown_connections 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf2930 0x563e1fc01650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fc03e80 0x563e1fc1b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/3504192437 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fbf1f80 0x563e1fc0f810 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3504192437 >> 172.21.15.7:0/3504192437 conn(0x563e1fbec8a0 msgr2=0x563e1fc00f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.454+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3504192437 shutdown_connections 2024-04-20T21:38:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.455+0000 7f65cfbe3740 1 -- 172.21.15.7:0/3504192437 wait complete. 2024-04-20T21:38:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.456+0000 7f65cfbe3740 1 -- start start 2024-04-20T21:38:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.456+0000 7f65cfbe3740 1 Processor -- start 2024-04-20T21:38:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 0x563e1fc0bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 0x563e1fc0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b2600 con 0x563e1fbf2930 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b3a70 con 0x563e1fbf1f80 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f65c81b2cd0 con 0x563e1fc03e80 2024-04-20T21:38:02.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65cfbe3740 10 client.0 _ll_register_callbacks cb 0x7f65c00f5d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65c6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65c5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 0x563e1fc0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65c6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc08cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:41450/0 (socket says 172.21.15.7:41450) 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.457+0000 7f65c6800640 1 -- 172.21.15.7:0/63561823 learned_addr learned my addr 172.21.15.7:0/63561823 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 0x563e1fc0bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 -- 172.21.15.7:0/63561823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 msgr2=0x563e1fc08cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc08cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 -- 172.21.15.7:0/63561823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 msgr2=0x563e1fc0c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 0x563e1fc0c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5e00640 1 -- 172.21.15.7:0/63561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1f9337d0 con 0x563e1fbf1f80 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.458+0000 7f65c5400640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 0x563e1fc0c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.459+0000 7f65c5e00640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 0x563e1fc0bd20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f65b0000c00 tx=0x7f65b0010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7e02d46be24f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.459+0000 7f65abe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.460+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0010910 con 0x563e1fbf1f80 2024-04-20T21:38:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.460+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f65b0010ab0 con 0x563e1fbf1f80 2024-04-20T21:38:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.460+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65b0020030 con 0x563e1fbf1f80 2024-04-20T21:38:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.461+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65b0010cb0 con 0x563e1fbf1f80 2024-04-20T21:38:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.461+0000 7f65cfbe3740 10 client.26135 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.461+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563e1fc7a5a0 con 0x563e1fbf1f80 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65b0020030 con 0x563e1fbf1f80 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f65b002d430 con 0x563e1fbf1f80 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 10 client.26135 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 20 client.26135 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc7acd0 0x563e1fc7d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 4 client.26135 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.462+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e1fb5e8e0 con 0x563e1fc7acd0 2024-04-20T21:38:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.463+0000 7f65c5400640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc7acd0 0x563e1fc7d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.464+0000 7f65abe00640 10 client.26135 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.464+0000 7f65c5400640 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc7acd0 0x563e1fc7d0f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0363d33b101cd53 server_cookie=2ff019483857db48 in_seq=0 out_seq=0 2024-04-20T21:38:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.464+0000 7f65abe00640 1 -- 172.21.15.7:0/63561823 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e1fb5e8e0 con 0x563e1fc7acd0 2024-04-20T21:38:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.464+0000 7f65abe00640 10 client.26135 handle_command_reply: tid=0 2024-04-20T21:38:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.464+0000 7f65cfbe3740 1 client.26135 shutdown 2024-04-20T21:38:02.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.466+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 msgr2=0x563e1fc0bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.466+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 0x563e1fc0bd20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f65b0000c00 tx=0x7f65b0010430 comp rx=0 tx=0).stop 2024-04-20T21:38:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.466+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 shutdown_connections 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.466+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563e1fc7acd0 0x563e1fc7d0f0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.466+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563e1fc03e80 0x563e1fc08cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563e1fbf2930 0x563e1fc0c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65abe00640 0 client.26135 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 1 --2- 172.21.15.7:0/63561823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563e1fbf1f80 0x563e1fc0bd20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 >> 172.21.15.7:0/63561823 conn(0x563e1fbec8a0 msgr2=0x563e1fbeb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 shutdown_connections 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 1 -- 172.21.15.7:0/63561823 wait complete. 2024-04-20T21:38:02.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.467+0000 7f65cfbe3740 20 client.26135 trim_cache size 0 max 16384 2024-04-20T21:38:02.467 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:38:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 msgr2=0x7f65ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 0x7f65ac0a6270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f65b8015c30 tx=0x7f65b80157b0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 msgr2=0x7f65c81a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f65c002c240 tx=0x7f65c00043b0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 shutdown_connections 2024-04-20T21:38:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f65ac0a3db0 0x7f65ac0a6270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c8114c00 0x7f65c81a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65c8109530 0x7f65c81a1400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 --2- 172.21.15.7:0/715599384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c806e240 0x7f65c81a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.470+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 >> 172.21.15.7:0/715599384 conn(0x7f65c806ce70 msgr2=0x7f65c8071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.471+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 shutdown_connections 2024-04-20T21:38:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:02.471+0000 7f65c7200640 1 -- 172.21.15.7:0/715599384 wait complete. 2024-04-20T21:38:03.503 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:38:03.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:38:03.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:38:03.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:03 smithi007 ceph-mon[16867]: pgmap v144: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 49 MiB/s wr, 9.95k op/s 2024-04-20T21:38:03.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:03 smithi016 ceph-mon[21409]: pgmap v144: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 49 MiB/s wr, 9.95k op/s 2024-04-20T21:38:03.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:03 smithi186 ceph-mon[22157]: pgmap v144: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 49 MiB/s wr, 9.95k op/s 2024-04-20T21:38:05.150 INFO:tasks.workunit.client.0.smithi007.stdout:Wrote -1 instead of 4096 bytes. 2024-04-20T21:38:05.150 INFO:tasks.workunit.client.0.smithi007.stdout:Probably out of disk space 2024-04-20T21:38:05.150 INFO:tasks.workunit.client.0.smithi007.stderr:write: Resource temporarily unavailable 2024-04-20T21:38:05.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:05 smithi016 ceph-mon[21409]: pgmap v145: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 70 MiB/s wr, 13.99k op/s 2024-04-20T21:38:05.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:05 smithi186 ceph-mon[22157]: pgmap v145: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 70 MiB/s wr, 13.99k op/s 2024-04-20T21:38:05.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:05 smithi007 ceph-mon[16867]: pgmap v145: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 70 MiB/s wr, 13.99k op/s 2024-04-20T21:38:06.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:38:06.045 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-20T21:38:06.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.709+0000 7f88be400640 1 -- 172.21.15.7:0/92435334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b810e9f0 msgr2=0x7f88b810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.709+0000 7f88be400640 1 --2- 172.21.15.7:0/92435334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b810e9f0 0x7f88b810edd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f88a00073b0 tx=0x7f88a002c7f0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 -- 172.21.15.7:0/92435334 shutdown_connections 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 --2- 172.21.15.7:0/92435334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b8109920 0x7f88b81158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 --2- 172.21.15.7:0/92435334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b81093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 --2- 172.21.15.7:0/92435334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b810e9f0 0x7f88b810edd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 -- 172.21.15.7:0/92435334 >> 172.21.15.7:0/92435334 conn(0x7f88b806d000 msgr2=0x7f88b806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 -- 172.21.15.7:0/92435334 shutdown_connections 2024-04-20T21:38:06.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.710+0000 7f88be400640 1 -- 172.21.15.7:0/92435334 wait complete. 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.711+0000 7f88be400640 1 Processor -- start 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.711+0000 7f88be400640 1 -- start start 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b811ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 0x7f88b811f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 0x7f88b8124c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8118690 con 0x7f88b8108f60 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8118510 con 0x7f88b810e9f0 2024-04-20T21:38:06.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88be400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8118810 con 0x7f88b8109920 2024-04-20T21:38:06.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88aea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 0x7f88b811f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88aea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 0x7f88b811f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50592/0 (socket says 172.21.15.7:50592) 2024-04-20T21:38:06.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88aea00640 1 -- 172.21.15.7:0/1612332877 learned_addr learned my addr 172.21.15.7:0/1612332877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:06.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.712+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 0x7f88b8124c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b811ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 -- 172.21.15.7:0/1612332877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 msgr2=0x7f88b811f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 0x7f88b811f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 -- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 msgr2=0x7f88b811ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b811ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 -- 172.21.15.7:0/1612332877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b8125520 con 0x7f88b810e9f0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b811ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.713+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 0x7f88b8124c60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f88b000ec50 tx=0x7f88b000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.714+0000 7f88b4c00640 1 -- 172.21.15.7:0/1612332877 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0003b70 con 0x7f88b810e9f0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.714+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b8125810 con 0x7f88b810e9f0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.714+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b811a290 con 0x7f88b810e9f0 2024-04-20T21:38:06.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.714+0000 7f88b4c00640 1 -- 172.21.15.7:0/1612332877 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b0003d10 con 0x7f88b810e9f0 2024-04-20T21:38:06.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.717+0000 7f88b4c00640 1 -- 172.21.15.7:0/1612332877 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0010670 con 0x7f88b810e9f0 2024-04-20T21:38:06.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.718+0000 7f88c103e740 1 Processor -- start 2024-04-20T21:38:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.722+0000 7f88b4c00640 1 -- 172.21.15.7:0/1612332877 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f88b0003ec0 con 0x7f88b810e9f0 2024-04-20T21:38:06.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.722+0000 7f88c103e740 1 -- start start 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523cfa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d05800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564523c5ed20 con 0x564523cf9f10 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564523c053e0 con 0x564523cfa830 2024-04-20T21:38:06.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.724+0000 7f88c103e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564523c5ace0 con 0x564523d03410 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.725+0000 7f88b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.725+0000 7f88b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58330/0 (socket says 172.21.15.7:58330) 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.725+0000 7f88b7400640 1 -- 172.21.15.7:0/1824012268 learned_addr learned my addr 172.21.15.7:0/1824012268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.726+0000 7f88aea00640 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d05800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.726+0000 7f88aea00640 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523cfa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.726+0000 7f88b4c00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 0x7f889c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.726+0000 7f88b4c00640 1 -- 172.21.15.7:0/1612332877 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f88b00d0f10 con 0x7f88b810e9f0 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.726+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 0x7f889c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.727+0000 7f88b7400640 1 -- 172.21.15.7:0/1824012268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 msgr2=0x564523d05800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.727+0000 7f88b7400640 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d05800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.727+0000 7f88b7400640 1 -- 172.21.15.7:0/1824012268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 msgr2=0x564523cfa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.727+0000 7f88b7400640 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523cfa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.727+0000 7f88b7400640 1 -- 172.21.15.7:0/1824012268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564523c24980 con 0x564523cfa830 2024-04-20T21:38:06.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.729+0000 7f88b7400640 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88a0000c00 tx=0x7f88a0005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=993ca14fd214f9da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.729+0000 7f88b7e00640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 0x7f889c0a6220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88b00a55e0 tx=0x7f88b000ef50 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.729+0000 7f88acc00640 1 -- 172.21.15.7:0/1824012268 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0044070 con 0x564523cfa830 2024-04-20T21:38:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.729+0000 7f88acc00640 1 -- 172.21.15.7:0/1824012268 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0007c50 con 0x564523cfa830 2024-04-20T21:38:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.729+0000 7f88acc00640 1 -- 172.21.15.7:0/1824012268 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0039040 con 0x564523cfa830 2024-04-20T21:38:06.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 -- 172.21.15.7:0/1824012268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 msgr2=0x564523d02ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88a0000c00 tx=0x7f88a0005760 comp rx=0 tx=0).stop 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 -- 172.21.15.7:0/1824012268 shutdown_connections 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cfa830 0x564523d02ed0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88a0000c00 tx=0x7f88a0005760 comp rx=0 tx=0).stop 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d05800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 --2- 172.21.15.7:0/1824012268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523cfa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.730+0000 7f88c103e740 1 -- 172.21.15.7:0/1824012268 >> 172.21.15.7:0/1824012268 conn(0x564523cf4640 msgr2=0x564523cf4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.731+0000 7f88c103e740 1 -- 172.21.15.7:0/1824012268 shutdown_connections 2024-04-20T21:38:06.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.731+0000 7f88c103e740 1 -- 172.21.15.7:0/1824012268 wait complete. 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.732+0000 7f88c103e740 1 -- start start 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.732+0000 7f88c103e740 1 Processor -- start 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 0x564523d009b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d00ef0 0x564523d0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564523d061a0 con 0x564523d03410 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564523c187a0 con 0x564523cf9f10 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564523d06320 con 0x564523d00ef0 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88c103e740 10 client.0 _ll_register_callbacks cb 0x564523c1ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:06.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.733+0000 7f88aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.734+0000 7f88aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:58346/0 (socket says 172.21.15.7:58346) 2024-04-20T21:38:06.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.734+0000 7f88aea00640 1 -- 172.21.15.7:0/4066861749 learned_addr learned my addr 172.21.15.7:0/4066861749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.734+0000 7f88aea00640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d00ef0 0x564523d0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.734+0000 7f88b7400640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 0x564523d009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 -- 172.21.15.7:0/4066861749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d00ef0 msgr2=0x564523d0f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d00ef0 0x564523d0f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 -- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 msgr2=0x564523d00470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 -- 172.21.15.7:0/4066861749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564523a397e0 con 0x564523d03410 2024-04-20T21:38:06.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88aea00640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88b7400640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 0x564523d009b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x564523cf5540 tx=0x7f88a002ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24419c478df06ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.735+0000 7f88a5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.736+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0044070 con 0x564523d03410 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.736+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f88a00043f0 con 0x564523d03410 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.736+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a0007c50 con 0x564523d03410 2024-04-20T21:38:06.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.737+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88a000adf0 con 0x564523d03410 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.737+0000 7f88c103e740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.737+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564523a99f70 con 0x564523d03410 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0007c50 con 0x564523d03410 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f88a004a6a0 con 0x564523d03410 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 20 client.15957 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d13af0 0x564523d15ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 4 client.15957 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:06.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.738+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564523c6c740 con 0x564523d13af0 2024-04-20T21:38:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.739+0000 7f88aea00640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d13af0 0x564523d15ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.739+0000 7f88aea00640 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d13af0 0x564523d15ee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f597c599a80c44c server_cookie=aea57d792b0f7a4 in_seq=0 out_seq=0 2024-04-20T21:38:06.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.739+0000 7f88a5600640 10 client.15957 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.740+0000 7f88a5600640 1 -- 172.21.15.7:0/4066861749 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564523c6c740 con 0x564523d13af0 2024-04-20T21:38:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.740+0000 7f88a5600640 10 client.15957 handle_command_reply: tid=0 2024-04-20T21:38:06.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.740+0000 7f88c103e740 1 client.15957 shutdown 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 msgr2=0x564523d009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 0x564523d009b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x564523cf5540 tx=0x7f88a002ce70 comp rx=0 tx=0).stop 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 shutdown_connections 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d13af0 0x564523d15ee0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d00ef0 0x564523d0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d03410 0x564523d009b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 --2- 172.21.15.7:0/4066861749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d00470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 >> 172.21.15.7:0/4066861749 conn(0x564523cf4640 msgr2=0x564523cf6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.741+0000 7f88a5600640 0 client.15957 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:06.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.742+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 shutdown_connections 2024-04-20T21:38:06.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.742+0000 7f88c103e740 1 -- 172.21.15.7:0/4066861749 wait complete. 2024-04-20T21:38:06.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.742+0000 7f88c103e740 20 client.15957 trim_cache size 0 max 16384 2024-04-20T21:38:06.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.761+0000 7f88c103e740 1 Processor -- start 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.761+0000 7f88c103e740 1 -- start start 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 0x564523d18f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d19440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8894010f20 con 0x564523cf9f10 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f889400e7e0 con 0x564523d00ef0 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88c103e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f889400c630 con 0x564523d03410 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88aea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88b7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 0x564523d18f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88aea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36646/0 (socket says 172.21.15.7:36646) 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88aea00640 1 -- 172.21.15.7:0/4003216192 learned_addr learned my addr 172.21.15.7:0/4003216192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88aea00640 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d19440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88b7400640 1 -- 172.21.15.7:0/4003216192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 msgr2=0x564523d19440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.762+0000 7f88b7400640 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d19440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88b7400640 1 -- 172.21.15.7:0/4003216192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 msgr2=0x564523d0f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88b7400640 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88b7400640 1 -- 172.21.15.7:0/4003216192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564523931170 con 0x564523d00ef0 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88aea00640 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88b7400640 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 0x564523d18f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a000af90 tx=0x7f88a0034460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fb20dd3469f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88a7e00640 1 -- 172.21.15.7:0/4003216192 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0056020 con 0x564523d00ef0 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88a7e00640 1 -- 172.21.15.7:0/4003216192 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0030050 con 0x564523d00ef0 2024-04-20T21:38:06.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.763+0000 7f88a7e00640 1 -- 172.21.15.7:0/4003216192 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0039030 con 0x564523d00ef0 2024-04-20T21:38:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- 172.21.15.7:0/4003216192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 msgr2=0x564523d18f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 0x564523d18f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a000af90 tx=0x7f88a0034460 comp rx=0 tx=0).stop 2024-04-20T21:38:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- 172.21.15.7:0/4003216192 shutdown_connections 2024-04-20T21:38:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523d00ef0 0x564523d18f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d19440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 --2- 172.21.15.7:0/4003216192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523cf9f10 0x564523d0f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- 172.21.15.7:0/4003216192 >> 172.21.15.7:0/4003216192 conn(0x564523cf4640 msgr2=0x564523cfed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- 172.21.15.7:0/4003216192 shutdown_connections 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- 172.21.15.7:0/4003216192 wait complete. 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.764+0000 7f88c103e740 1 -- start start 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 Processor -- start 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d17450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f889400cca0 con 0x564523d00ef0 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f889400e110 con 0x564523cf9f10 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f889400d370 con 0x564523d03410 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88c103e740 10 client.0 _ll_register_callbacks cb 0x7f88a0042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36648/0 (socket says 172.21.15.7:36648) 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88b7400640 1 -- 172.21.15.7:0/2279174554 learned_addr learned my addr 172.21.15.7:0/2279174554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88aea00640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.765+0000 7f88aea00640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d17450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 -- 172.21.15.7:0/2279174554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 msgr2=0x564523d17450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d17450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 -- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 msgr2=0x564523d169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 -- 172.21.15.7:0/2279174554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564523d78da0 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88aea00640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88b7400640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88a004aaa0 tx=0x7f88a0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615ce74b2f37e8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88a5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0056070 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f88a0003d80 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a0030050 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88a0044040 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88c103e740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.766+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564523d7a0e0 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0030050 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f88a003fa70 con 0x564523d00ef0 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:06.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 20 client.15963 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d7a750 0x564523d7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 4 client.15963 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564523c665b0 con 0x564523d7a750 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.767+0000 7f88aea00640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d7a750 0x564523d7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.768+0000 7f88aea00640 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d7a750 0x564523d7cb70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa910fda290f37d1 server_cookie=475101be91504c51 in_seq=0 out_seq=0 2024-04-20T21:38:06.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.768+0000 7f88a5600640 10 client.15963 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88a5600640 1 -- 172.21.15.7:0/2279174554 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564523c665b0 con 0x564523d7a750 2024-04-20T21:38:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88a5600640 10 client.15963 handle_command_reply: tid=0 2024-04-20T21:38:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 client.15963 shutdown 2024-04-20T21:38:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 msgr2=0x564523d16f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88a004aaa0 tx=0x7f88a0005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 shutdown_connections 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564523d7a750 0x564523d7cb70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564523d03410 0x564523d17450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564523d00ef0 0x564523d16f10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 --2- 172.21.15.7:0/2279174554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564523cf9f10 0x564523d169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 >> 172.21.15.7:0/2279174554 conn(0x564523cf4640 msgr2=0x564523cf5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88a5600640 0 client.15963 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 shutdown_connections 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 1 -- 172.21.15.7:0/2279174554 wait complete. 2024-04-20T21:38:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.769+0000 7f88c103e740 20 client.15963 trim_cache size 0 max 16384 2024-04-20T21:38:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:38:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:38:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "da1e69ee-0554-4a14-96f6-853376a086fa", 2024-04-20T21:38:06.773 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:38:06.773 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 msgr2=0x7f889c0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 0x7f889c0a6220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88b00a55e0 tx=0x7f88b000ef50 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 msgr2=0x7f88b8124c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 0x7f88b8124c60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f88b000ec50 tx=0x7f88b000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 shutdown_connections 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f889c0a3d60 0x7f889c0a6220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88b810e9f0 0x7f88b8124c60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f88b8109920 0x7f88b811f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 --2- 172.21.15.7:0/1612332877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88b8108f60 0x7f88b811ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 >> 172.21.15.7:0/1612332877 conn(0x7f88b806d000 msgr2=0x7f88b810b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:06.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 shutdown_connections 2024-04-20T21:38:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:06.770+0000 7f88be400640 1 -- 172.21.15.7:0/1612332877 wait complete. 2024-04-20T21:38:06.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:06.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 60 MiB/s wr, 11.80k op/s 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:07.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 60 MiB/s wr, 11.80k op/s 2024-04-20T21:38:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:07.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:07.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:07.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 60 MiB/s wr, 11.80k op/s 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.447+0000 7f1b3d000640 1 -- 172.21.15.7:0/3068881870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 msgr2=0x7f1b38104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.447+0000 7f1b3d000640 1 --2- 172.21.15.7:0/3068881870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b38104780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009900 tx=0x7f1b2802bc40 comp rx=0 tx=0).stop 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 -- 172.21.15.7:0/3068881870 shutdown_connections 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 --2- 172.21.15.7:0/3068881870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b38104d50 0x7f1b38108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 --2- 172.21.15.7:0/3068881870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 0x7f1b38111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 --2- 172.21.15.7:0/3068881870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b38104780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 -- 172.21.15.7:0/3068881870 >> 172.21.15.7:0/3068881870 conn(0x7f1b380ffbe0 msgr2=0x7f1b38102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 -- 172.21.15.7:0/3068881870 shutdown_connections 2024-04-20T21:38:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.448+0000 7f1b3d000640 1 -- 172.21.15.7:0/3068881870 wait complete. 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 Processor -- start 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 -- start start 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 0x7f1b381a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 0x7f1b3819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38114530 con 0x7f1b381043a0 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381143b0 con 0x7f1b3819a8f0 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.449+0000 7f1b3d000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381146b0 con 0x7f1b38109470 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36672/0 (socket says 172.21.15.7:36672) 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 -- 172.21.15.7:0/1296635284 learned_addr learned my addr 172.21.15.7:0/1296635284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b36800640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 0x7f1b3819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 0x7f1b381a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 -- 172.21.15.7:0/1296635284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 msgr2=0x7f1b381a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 0x7f1b381a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 -- 172.21.15.7:0/1296635284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 msgr2=0x7f1b3819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 0x7f1b3819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 -- 172.21.15.7:0/1296635284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3819b410 con 0x7f1b381043a0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b36800640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 0x7f1b3819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.450+0000 7f1b35e00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006010 tx=0x7f1b2802cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.451+0000 7f1b2ea00640 1 -- 172.21.15.7:0/1296635284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28041070 con 0x7f1b381043a0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.451+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3819b6a0 con 0x7f1b381043a0 2024-04-20T21:38:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.451+0000 7f1b2ea00640 1 -- 172.21.15.7:0/1296635284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b28004450 con 0x7f1b381043a0 2024-04-20T21:38:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.451+0000 7f1b2ea00640 1 -- 172.21.15.7:0/1296635284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28003e30 con 0x7f1b381043a0 2024-04-20T21:38:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.451+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a7840 con 0x7f1b381043a0 2024-04-20T21:38:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.455+0000 7f1b3fd99740 1 Processor -- start 2024-04-20T21:38:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 -- start start 2024-04-20T21:38:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b2ea00640 1 -- 172.21.15.7:0/1296635284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1b28044070 con 0x7f1b381043a0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ed0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ed06a0 0x561417ed2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561417e2d150 con 0x561417ec7ac0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561417dd3370 con 0x561417ed06a0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.456+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561417e29110 con 0x561417ec71a0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ed0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50672/0 (socket says 172.21.15.7:50672) 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 -- 172.21.15.7:0/2246173420 learned_addr learned my addr 172.21.15.7:0/2246173420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ed06a0 0x561417ed2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 -- 172.21.15.7:0/2246173420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ed06a0 msgr2=0x561417ed2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ed06a0 0x561417ed2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 -- 172.21.15.7:0/2246173420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 msgr2=0x561417ed0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ed0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 -- 172.21.15.7:0/2246173420 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561417c097d0 con 0x561417ec71a0 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b35e00640 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ed0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.457+0000 7f1b36800640 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1b30010710 tx=0x7f1b30010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba8c9785de3445b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b27400640 1 -- 172.21.15.7:0/2246173420 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000e8d0 con 0x561417ec71a0 2024-04-20T21:38:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b27400640 1 -- 172.21.15.7:0/2246173420 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30012450 con 0x561417ec71a0 2024-04-20T21:38:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b27400640 1 -- 172.21.15.7:0/2246173420 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3001c650 con 0x561417ec71a0 2024-04-20T21:38:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b2ea00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 0x7f1b1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b2ea00640 1 -- 172.21.15.7:0/1296635284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1b280f21f0 con 0x7f1b381043a0 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.458+0000 7f1b35e00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 0x7f1b1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2246173420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 msgr2=0x561417ec7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1b30010710 tx=0x7f1b30010bd0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2246173420 shutdown_connections 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ed06a0 0x561417ed2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ed0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2246173420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec71a0 0x561417ec7580 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2246173420 >> 172.21.15.7:0/2246173420 conn(0x561417ec18d0 msgr2=0x561417ec1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2246173420 shutdown_connections 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b35e00640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 0x7f1b1c0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x561417ec6c70 tx=0x7f1b2802bc40 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.459+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2246173420 wait complete. 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 -- start start 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 Processor -- start 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ecdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561417ed3490 con 0x561417ec7ac0 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561417de6740 con 0x561417ec71a0 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.460+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561417ed3610 con 0x561417ed06a0 2024-04-20T21:38:08.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b3fd99740 10 client.0 _ll_register_callbacks cb 0x561417de1b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b2fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ecdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50676/0 (socket says 172.21.15.7:50676) 2024-04-20T21:38:08.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 -- 172.21.15.7:0/935835664 learned_addr learned my addr 172.21.15.7:0/935835664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 -- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 msgr2=0x561417ecd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 -- 172.21.15.7:0/935835664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 msgr2=0x561417ecdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ecdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.461+0000 7f1b36800640 1 -- 172.21.15.7:0/935835664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561417b3f3f0 con 0x561417ed06a0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b36800640 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b300092f0 tx=0x7f1b30012db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606993a5e514097a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30020030 con 0x561417ed06a0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1b3000ed80 con 0x561417ed06a0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b30012450 con 0x561417ed06a0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b30023050 con 0x561417ed06a0 2024-04-20T21:38:08.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b3fd99740 10 client.26143 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.462+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561417aff0f0 con 0x561417ed06a0 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b30012450 con 0x561417ed06a0 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1b3000f040 con 0x561417ed06a0 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 10 client.26143 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 10 client.26143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 20 client.26143 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417ee0970 0x561417ee2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 4 client.26143 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561417e3ab70 con 0x561417ee0970 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.463+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417ee0970 0x561417ee2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.464+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417ee0970 0x561417ee2d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b608f780b5415634 server_cookie=756699b570520fb in_seq=0 out_seq=0 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.464+0000 7f1b1be00640 10 client.26143 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:08.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b1be00640 1 -- 172.21.15.7:0/935835664 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561417e3ab70 con 0x561417ee0970 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b1be00640 10 client.26143 handle_command_reply: tid=0 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 client.26143 shutdown 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 msgr2=0x561417ece1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1b300092f0 tx=0x7f1b30012db0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 shutdown_connections 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417ee0970 0x561417ee2d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ece1b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec7ac0 0x561417ecdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b1be00640 0 client.26143 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/935835664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.465+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 >> 172.21.15.7:0/935835664 conn(0x561417ec18d0 msgr2=0x561417ec3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.466+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 shutdown_connections 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.466+0000 7f1b3fd99740 1 -- 172.21.15.7:0/935835664 wait complete. 2024-04-20T21:38:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.466+0000 7f1b3fd99740 20 client.26143 trim_cache size 0 max 16384 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 Processor -- start 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 -- start start 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ed06a0 0x561417edc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381b26a0 con 0x561417ed06a0 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381aff60 con 0x561417ec71a0 2024-04-20T21:38:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.485+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381addd0 con 0x561417ec7ac0 2024-04-20T21:38:08.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b36800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ed06a0 0x561417edc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50678/0 (socket says 172.21.15.7:50678) 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 -- 172.21.15.7:0/114655554 learned_addr learned my addr 172.21.15.7:0/114655554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 -- 172.21.15.7:0/114655554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 msgr2=0x561417ecfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 -- 172.21.15.7:0/114655554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ed06a0 msgr2=0x561417edc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ed06a0 0x561417edc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 -- 172.21.15.7:0/114655554 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561417ec2660 con 0x561417ec7ac0 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.486+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b38100100 tx=0x7f1b20005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ea1eb3856685b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.487+0000 7f1b26a00640 1 -- 172.21.15.7:0/114655554 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20023030 con 0x561417ec7ac0 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.487+0000 7f1b26a00640 1 -- 172.21.15.7:0/114655554 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2001f910 con 0x561417ec7ac0 2024-04-20T21:38:08.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.487+0000 7f1b26a00640 1 -- 172.21.15.7:0/114655554 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20004a00 con 0x561417ec7ac0 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.487+0000 7f1b3fd99740 1 -- 172.21.15.7:0/114655554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 msgr2=0x561417ecc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.487+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1b38100100 tx=0x7f1b20005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 -- 172.21.15.7:0/114655554 shutdown_connections 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ed06a0 0x561417edc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ec7ac0 0x561417ecc670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/114655554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec71a0 0x561417ecfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 -- 172.21.15.7:0/114655554 >> 172.21.15.7:0/114655554 conn(0x561417ec18d0 msgr2=0x561417ec08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 -- 172.21.15.7:0/114655554 shutdown_connections 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 -- 172.21.15.7:0/114655554 wait complete. 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 -- start start 2024-04-20T21:38:08.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.488+0000 7f1b3fd99740 1 Processor -- start 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 0x561417ed8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 0x561417ed4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ed4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381ae420 con 0x561417ec71a0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381af890 con 0x561417ec7ac0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1b381aeaf0 con 0x561417ed06a0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b3fd99740 10 client.0 _ll_register_callbacks cb 0x7f1b30016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b36800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 0x561417ed8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b36800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 0x561417ed8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36698/0 (socket says 172.21.15.7:36698) 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b36800640 1 -- 172.21.15.7:0/2765933384 learned_addr learned my addr 172.21.15.7:0/2765933384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 0x561417ed4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.489+0000 7f1b36800640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ed4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 -- 172.21.15.7:0/2765933384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 msgr2=0x561417ed4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ed4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 -- 172.21.15.7:0/2765933384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 msgr2=0x561417ed8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:08.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 0x561417ed8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 -- 172.21.15.7:0/2765933384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561417ed4e40 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b2fe00640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 0x561417ed4090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x561417ece7c0 tx=0x7f1b20009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c3f3026fffd462 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b1be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20005220 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b2001f910 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.490+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1b2001f430 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b20035020 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 10 client.26171 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561417f46cf0 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2001f910 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1b200334b0 con 0x561417ec7ac0 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 10 client.26171 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:08.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 20 client.26171 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417f47390 0x561417f497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 4 client.26171 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.491+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561417e349e0 con 0x561417f47390 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b36800640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417f47390 0x561417f497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b36800640 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417f47390 0x561417f497b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f58e2eaebd23783 server_cookie=c80c0b247d3243bd in_seq=0 out_seq=0 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b1be00640 10 client.26171 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b1be00640 1 -- 172.21.15.7:0/2765933384 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561417e349e0 con 0x561417f47390 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b1be00640 10 client.26171 handle_command_reply: tid=0 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b3fd99740 1 client.26171 shutdown 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 msgr2=0x561417ed4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.492+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 0x561417ed4090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x561417ece7c0 tx=0x7f1b20009ba0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 shutdown_connections 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561417ec7ac0 0x561417ed4090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561417f47390 0x561417f497b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561417ed06a0 0x561417ed4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 --2- 172.21.15.7:0/2765933384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561417ec71a0 0x561417ed8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b1be00640 0 client.26171 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 >> 172.21.15.7:0/2765933384 conn(0x561417ec18d0 msgr2=0x561417ecd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 shutdown_connections 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 1 -- 172.21.15.7:0/2765933384 wait complete. 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.493+0000 7f1b3fd99740 20 client.26171 trim_cache size 0 max 16384 2024-04-20T21:38:08.495 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "da1e69ee-0554-4a14-96f6-853376a086fa": { 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "da1e69ee-0554-4a14-96f6-853376a086fa", 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:38:08.496 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 msgr2=0x7f1b1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 0x7f1b1c0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x561417ec6c70 tx=0x7f1b2802bc40 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 msgr2=0x7f1b381a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006010 tx=0x7f1b2802cb40 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 shutdown_connections 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1b1c0a3d20 0x7f1b1c0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b3819a8f0 0x7f1b3819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b38109470 0x7f1b381a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 --2- 172.21.15.7:0/1296635284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b381043a0 0x7f1b381a06f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 >> 172.21.15.7:0/1296635284 conn(0x7f1b380ffbe0 msgr2=0x7f1b38100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:08.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 shutdown_connections 2024-04-20T21:38:08.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:08.494+0000 7f1b3d000640 1 -- 172.21.15.7:0/1296635284 wait complete. 2024-04-20T21:38:08.511 INFO:tasks.cephfs.filesystem:scrub status for tag:da1e69ee-0554-4a14-96f6-853376a086fa - {'path': '/', 'tag': 'da1e69ee-0554-4a14-96f6-853376a086fa', 'options': 'recursive,force'} 2024-04-20T21:38:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:38:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:08 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:38:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:08.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:08.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:08 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:08 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:09.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:09 smithi016 ceph-mon[21409]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 65 MiB/s wr, 12.69k op/s 2024-04-20T21:38:09.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:09 smithi186 ceph-mon[22157]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 65 MiB/s wr, 12.69k op/s 2024-04-20T21:38:09.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:09 smithi007 ceph-mon[16867]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 65 MiB/s wr, 12.69k op/s 2024-04-20T21:38:11.002 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-04-20T21:38:11.259 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=32523486170d42bba4ac50c0983dab44 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi007 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-04-20T21:38:11.262 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-20T21:38:11.274 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-20T21:38:11.275 INFO:teuthology.task.exec:Executing custom commands... 2024-04-20T21:38:11.275 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:38:11.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-20T21:38:11.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-20T21:38:11.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:11 smithi016 ceph-mon[21409]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 8.05k op/s 2024-04-20T21:38:11.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:11 smithi186 ceph-mon[22157]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 8.05k op/s 2024-04-20T21:38:11.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:11 smithi007 ceph-mon[16867]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 42 MiB/s wr, 8.05k op/s 2024-04-20T21:38:13.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:13 smithi186 ceph-mon[22157]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 7.88k op/s 2024-04-20T21:38:13.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:13 smithi007 ceph-mon[16867]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 7.88k op/s 2024-04-20T21:38:13.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:13 smithi016 ceph-mon[21409]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 42 MiB/s wr, 7.88k op/s 2024-04-20T21:38:15.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:15 smithi186 ceph-mon[22157]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 MiB/s wr, 10.18k op/s 2024-04-20T21:38:15.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:15 smithi016 ceph-mon[21409]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 MiB/s wr, 10.18k op/s 2024-04-20T21:38:15.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:15 smithi007 ceph-mon[16867]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 57 MiB/s wr, 10.18k op/s 2024-04-20T21:38:17.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:17 smithi186 ceph-mon[22157]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 5.79k op/s 2024-04-20T21:38:17.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:17.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:17.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:17 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:17.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:17 smithi007 ceph-mon[16867]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 5.79k op/s 2024-04-20T21:38:17.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:17.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:17.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:17 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:17 smithi016 ceph-mon[21409]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 34 MiB/s wr, 5.79k op/s 2024-04-20T21:38:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:17 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:19.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:19 smithi007 ceph-mon[16867]: pgmap v152: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 MiB/s wr, 7.72k op/s 2024-04-20T21:38:19.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:19 smithi016 ceph-mon[21409]: pgmap v152: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 MiB/s wr, 7.72k op/s 2024-04-20T21:38:19.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:19 smithi186 ceph-mon[22157]: pgmap v152: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 45 MiB/s wr, 7.72k op/s 2024-04-20T21:38:21.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:21 smithi007 ceph-mon[16867]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 7.43k op/s 2024-04-20T21:38:21.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:21 smithi016 ceph-mon[21409]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 7.43k op/s 2024-04-20T21:38:21.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:21 smithi186 ceph-mon[22157]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 44 MiB/s wr, 7.43k op/s 2024-04-20T21:38:23.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:23 smithi007 ceph-mon[16867]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 MiB/s wr, 9.08k op/s 2024-04-20T21:38:23.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:23 smithi016 ceph-mon[21409]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 MiB/s wr, 9.08k op/s 2024-04-20T21:38:23.972 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:23 smithi186 ceph-mon[22157]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 MiB/s wr, 9.08k op/s 2024-04-20T21:38:25.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:25 smithi007 ceph-mon[16867]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 MiB/s wr, 10.51k op/s 2024-04-20T21:38:25.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:25 smithi016 ceph-mon[21409]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 MiB/s wr, 10.51k op/s 2024-04-20T21:38:25.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:25 smithi186 ceph-mon[22157]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 60 MiB/s wr, 10.51k op/s 2024-04-20T21:38:27.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:27 smithi007 ceph-mon[16867]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 MiB/s wr, 8.22k op/s 2024-04-20T21:38:27.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:27 smithi016 ceph-mon[21409]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 MiB/s wr, 8.22k op/s 2024-04-20T21:38:27.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:27 smithi186 ceph-mon[22157]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 MiB/s wr, 8.22k op/s 2024-04-20T21:38:29.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:29 smithi007 ceph-mon[16867]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-20T21:38:29.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:29 smithi016 ceph-mon[21409]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-20T21:38:29.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:29 smithi186 ceph-mon[22157]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-20T21:38:31.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:31 smithi007 ceph-mon[16867]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 10.03k op/s 2024-04-20T21:38:31.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:31 smithi016 ceph-mon[21409]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 10.03k op/s 2024-04-20T21:38:31.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:31 smithi186 ceph-mon[22157]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 55 MiB/s wr, 10.03k op/s 2024-04-20T21:38:33.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:33 smithi016 ceph-mon[21409]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 9.44k op/s 2024-04-20T21:38:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:33 smithi186 ceph-mon[22157]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 9.44k op/s 2024-04-20T21:38:34.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:33 smithi007 ceph-mon[16867]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 52 MiB/s wr, 9.44k op/s 2024-04-20T21:38:35.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:35 smithi016 ceph-mon[21409]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 45 MiB/s wr, 8.23k op/s 2024-04-20T21:38:35.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:35 smithi186 ceph-mon[22157]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 45 MiB/s wr, 8.23k op/s 2024-04-20T21:38:36.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:35 smithi007 ceph-mon[16867]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 45 MiB/s wr, 8.23k op/s 2024-04-20T21:38:37.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:37 smithi016 ceph-mon[21409]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 6.95k op/s 2024-04-20T21:38:37.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:37 smithi186 ceph-mon[22157]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 6.95k op/s 2024-04-20T21:38:38.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:37 smithi007 ceph-mon[16867]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 MiB/s wr, 6.95k op/s 2024-04-20T21:38:38.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:38.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 -- 172.21.15.7:0/3127590825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076db0 msgr2=0x7faf9c1126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 --2- 172.21.15.7:0/3127590825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076db0 0x7faf9c1126a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faf84009890 tx=0x7faf8402bb70 comp rx=0 tx=0).stop 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 -- 172.21.15.7:0/3127590825 shutdown_connections 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 --2- 172.21.15.7:0/3127590825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076db0 0x7faf9c1126a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 --2- 172.21.15.7:0/3127590825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c112be0 0x7faf9c114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 --2- 172.21.15.7:0/3127590825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c076490 0x7faf9c076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 -- 172.21.15.7:0/3127590825 >> 172.21.15.7:0/3127590825 conn(0x7faf9c0ffa30 msgr2=0x7faf9c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 -- 172.21.15.7:0/3127590825 shutdown_connections 2024-04-20T21:38:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.798+0000 7fafa0c00640 1 -- 172.21.15.7:0/3127590825 wait complete. 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.800+0000 7fafa0c00640 1 Processor -- start 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.800+0000 7fafa0c00640 1 -- start start 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 0x7faf9c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 0x7faf9c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c118730 con 0x7faf9c112be0 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1185b0 con 0x7faf9c076490 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7fafa0c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1188b0 con 0x7faf9c076db0 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57688/0 (socket says 172.21.15.7:57688) 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7faf9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 0x7faf9c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7faf9be00640 1 -- 172.21.15.7:0/992785051 learned_addr learned my addr 172.21.15.7:0/992785051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.801+0000 7faf93e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 0x7faf9c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 -- 172.21.15.7:0/992785051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 msgr2=0x7faf9c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 0x7faf9c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 -- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 msgr2=0x7faf9c1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 -- 172.21.15.7:0/992785051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c1a3d70 con 0x7faf9c076db0 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf9be00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf9b400640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 0x7faf9c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf93e00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 0x7faf9c19f2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf840057c0 tx=0x7faf84004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf98c00640 1 -- 172.21.15.7:0/992785051 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84041020 con 0x7faf9c076db0 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c1a4000 con 0x7faf9c076db0 2024-04-20T21:38:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf98c00640 1 -- 172.21.15.7:0/992785051 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8402c9f0 con 0x7faf9c076db0 2024-04-20T21:38:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.802+0000 7faf98c00640 1 -- 172.21.15.7:0/992785051 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84039040 con 0x7faf9c076db0 2024-04-20T21:38:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.803+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c1ab8f0 con 0x7faf9c076db0 2024-04-20T21:38:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 Processor -- start 2024-04-20T21:38:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 -- start start 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7faf98c00640 1 -- 172.21.15.7:0/992785051 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7faf84044020 con 0x7faf9c076db0 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1ca0 0x564daebba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebba880 0x564daebbcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564daeb17150 con 0x564daebb1380 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564daeabd370 con 0x564daebb1ca0 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7fafa4e41740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564daeb13110 con 0x564daebba880 2024-04-20T21:38:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57694/0 (socket says 172.21.15.7:57694) 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7faf9be00640 1 -- 172.21.15.7:0/523218536 learned_addr learned my addr 172.21.15.7:0/523218536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.806+0000 7faf93e00640 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebba880 0x564daebbcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9b400640 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1ca0 0x564daebba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 -- 172.21.15.7:0/523218536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebba880 msgr2=0x564daebbcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebba880 0x564daebbcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 -- 172.21.15.7:0/523218536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1ca0 msgr2=0x564daebba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1ca0 0x564daebba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 -- 172.21.15.7:0/523218536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dae8f37d0 con 0x564daebb1380 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.807+0000 7faf9be00640 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e800 tx=0x7faf9400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd428362de7906a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf8be00640 1 -- 172.21.15.7:0/523218536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94011bc0 con 0x564daebb1380 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf8be00640 1 -- 172.21.15.7:0/523218536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf94011d60 con 0x564daebb1380 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf8be00640 1 -- 172.21.15.7:0/523218536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9400f620 con 0x564daebb1380 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf98c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 0x7faf800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf98c00640 1 -- 172.21.15.7:0/992785051 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7faf840f1e90 con 0x7faf9c076db0 2024-04-20T21:38:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7faf9b400640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 0x7faf800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7fafa4e41740 1 -- 172.21.15.7:0/523218536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 msgr2=0x564daebb1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.808+0000 7fafa4e41740 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faf9400e800 tx=0x7faf9400ecc0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 -- 172.21.15.7:0/523218536 shutdown_connections 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1ca0 0x564daebba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebba880 0x564daebbcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 --2- 172.21.15.7:0/523218536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1380 0x564daebb1760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 -- 172.21.15.7:0/523218536 >> 172.21.15.7:0/523218536 conn(0x564daebabab0 msgr2=0x564daebabe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 -- 172.21.15.7:0/523218536 shutdown_connections 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 -- 172.21.15.7:0/523218536 wait complete. 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7faf9b400640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 0x7faf800a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x564daebb0e50 tx=0x7faf8c009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 -- start start 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.809+0000 7fafa4e41740 1 Processor -- start 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1380 0x564daebb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1ca0 0x564daebb7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564daebbd670 con 0x564daebba880 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564daead0740 con 0x564daebb1380 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564daebbd7f0 con 0x564daebb1ca0 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7fafa4e41740 10 client.0 _ll_register_callbacks cb 0x564daeacbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7faf9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57706/0 (socket says 172.21.15.7:57706) 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7faf9be00640 1 -- 172.21.15.7:0/2016619570 learned_addr learned my addr 172.21.15.7:0/2016619570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7faf9b400640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1380 0x564daebb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.810+0000 7faf93e00640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1ca0 0x564daebb7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 -- 172.21.15.7:0/2016619570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1ca0 msgr2=0x564daebb7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1ca0 0x564daebb7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 -- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1380 msgr2=0x564daebb7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1380 0x564daebb7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 -- 172.21.15.7:0/2016619570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dae8293f0 con 0x564daebba880 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf9be00640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf9401abd0 tx=0x7faf9401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fd027bb36a3f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94028030 con 0x564daebba880 2024-04-20T21:38:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf94011d60 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.811+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7faf94004b40 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf94032050 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7fafa4e41740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564dae7e90f0 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf94011d60 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7faf94023030 con 0x564daebba880 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7fafa4e41740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7fafa4e41740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.812+0000 7fafa4e41740 20 client.15993 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.813+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daebcac60 0x564daebcd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.813+0000 7fafa4e41740 4 client.15993 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.813+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564daeb24b70 con 0x564daebcac60 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.813+0000 7faf9b400640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daebcac60 0x564daebcd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.814+0000 7faf9b400640 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daebcac60 0x564daebcd050 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938e16f25cd31c7f server_cookie=4b87efe6650fa034 in_seq=0 out_seq=0 2024-04-20T21:38:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.814+0000 7faf88c00640 10 client.15993 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7faf88c00640 1 -- 172.21.15.7:0/2016619570 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564daeb24b70 con 0x564daebcac60 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7faf88c00640 10 client.15993 handle_command_reply: tid=0 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 client.15993 shutdown 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 msgr2=0x564daebb83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faf9401abd0 tx=0x7faf9401ac00 comp rx=0 tx=0).stop 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 shutdown_connections 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daebcac60 0x564daebcd050 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebba880 0x564daebb83a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1ca0 0x564daebb7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 --2- 172.21.15.7:0/2016619570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebb1380 0x564daebb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 >> 172.21.15.7:0/2016619570 conn(0x564daebabab0 msgr2=0x564daebad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7faf88c00640 0 client.15993 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 shutdown_connections 2024-04-20T21:38:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 1 -- 172.21.15.7:0/2016619570 wait complete. 2024-04-20T21:38:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.815+0000 7fafa4e41740 20 client.15993 trim_cache size 0 max 16384 2024-04-20T21:38:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 Processor -- start 2024-04-20T21:38:38.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 -- start start 2024-04-20T21:38:38.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebc6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b68c0 con 0x564daebb1ca0 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b4180 con 0x564daebba880 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7fafa4e41740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b1ff0 con 0x564daebb1380 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7faf9be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.835+0000 7faf9b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40888/0 (socket says 172.21.15.7:40888) 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 -- 172.21.15.7:0/3437218373 learned_addr learned my addr 172.21.15.7:0/3437218373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf93e00640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebc6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 -- 172.21.15.7:0/3437218373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 msgr2=0x564daebccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 -- 172.21.15.7:0/3437218373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 msgr2=0x564daebc6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebc6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 -- 172.21.15.7:0/3437218373 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564daebb1ad0 con 0x564daebb1380 2024-04-20T21:38:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf93e00640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebc6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9be00640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:38.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.836+0000 7faf9b400640 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf8c004400 tx=0x7faf8c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7802ed38303e3f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7faf8b400640 1 -- 172.21.15.7:0/3437218373 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c02a4f0 con 0x564daebb1380 2024-04-20T21:38:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7faf8b400640 1 -- 172.21.15.7:0/3437218373 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8c02f070 con 0x564daebb1380 2024-04-20T21:38:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7faf8b400640 1 -- 172.21.15.7:0/3437218373 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c01e630 con 0x564daebb1380 2024-04-20T21:38:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 -- 172.21.15.7:0/3437218373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 msgr2=0x564daebba050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faf8c004400 tx=0x7faf8c01b000 comp rx=0 tx=0).stop 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 -- 172.21.15.7:0/3437218373 shutdown_connections 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebc6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 --2- 172.21.15.7:0/3437218373 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebba050 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 -- 172.21.15.7:0/3437218373 >> 172.21.15.7:0/3437218373 conn(0x564daebabab0 msgr2=0x564daebb78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 -- 172.21.15.7:0/3437218373 shutdown_connections 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.837+0000 7fafa4e41740 1 -- 172.21.15.7:0/3437218373 wait complete. 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.838+0000 7fafa4e41740 1 -- start start 2024-04-20T21:38:38.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.838+0000 7fafa4e41740 1 Processor -- start 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebc2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebb46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b2640 con 0x564daebb1ca0 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b3ab0 con 0x564daebba880 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1b2d10 con 0x564daebb1380 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7fafa4e41740 10 client.0 _ll_register_callbacks cb 0x7faf8c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf9b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebc2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57722/0 (socket says 172.21.15.7:57722) 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 -- 172.21.15.7:0/1924393702 learned_addr learned my addr 172.21.15.7:0/1924393702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf9be00640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebb46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 -- 172.21.15.7:0/1924393702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 msgr2=0x564daebc2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebc2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 -- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 msgr2=0x564daebb46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebb46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf93e00640 1 -- 172.21.15.7:0/1924393702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564daebb4d20 con 0x564daebb1ca0 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.839+0000 7faf9be00640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebb46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:38.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf93e00640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf8402cbc0 tx=0x7faf840f5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c484b77cb845198e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf840ba1d0 con 0x564daebb1ca0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7faf840be190 con 0x564daebb1ca0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8c02f070 con 0x564daebb1ca0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faf840d6030 con 0x564daebb1ca0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7fafa4e41740 10 client.15999 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564daec30fd0 con 0x564daebb1ca0 2024-04-20T21:38:38.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf8c02f070 con 0x564daebb1ca0 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.840+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7faf8403c030 con 0x564daebb1ca0 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 20 client.15999 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daec317a0 0x564daec33bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 4 client.15999 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564daeb1e9e0 con 0x564daec317a0 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.841+0000 7faf9b400640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daec317a0 0x564daec33bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7faf9b400640 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daec317a0 0x564daec33bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67e29826fb92351 server_cookie=3b0685ad664bf18 in_seq=0 out_seq=0 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7faf88c00640 10 client.15999 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7faf88c00640 1 -- 172.21.15.7:0/1924393702 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564daeb1e9e0 con 0x564daec317a0 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7faf88c00640 10 client.15999 handle_command_reply: tid=0 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 client.15999 shutdown 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 msgr2=0x564daebb4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf8402cbc0 tx=0x7faf840f5530 comp rx=0 tx=0).stop 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 shutdown_connections 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564daebba880 0x564daebb46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564daec317a0 0x564daec33bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564daebb1ca0 0x564daebb4170 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 --2- 172.21.15.7:0/1924393702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564daebb1380 0x564daebc2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7faf88c00640 0 client.15999 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 >> 172.21.15.7:0/1924393702 conn(0x564daebabab0 msgr2=0x564daebb80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 shutdown_connections 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 1 -- 172.21.15.7:0/1924393702 wait complete. 2024-04-20T21:38:38.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.842+0000 7fafa4e41740 20 client.15999 trim_cache size 0 max 16384 2024-04-20T21:38:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:38:38.850 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:38:38.850 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:38:38.850 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:38:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 msgr2=0x7faf800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 0x7faf800a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x564daebb0e50 tx=0x7faf8c009260 comp rx=0 tx=0).stop 2024-04-20T21:38:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 msgr2=0x7faf9c19f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:38.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 0x7faf9c19f2c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf840057c0 tx=0x7faf84004360 comp rx=0 tx=0).stop 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 shutdown_connections 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf9c112be0 0x7faf9c1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faf9c076db0 0x7faf9c19f2c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7faf800a3da0 0x7faf800a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 --2- 172.21.15.7:0/992785051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf9c076490 0x7faf9c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 >> 172.21.15.7:0/992785051 conn(0x7faf9c0ffa30 msgr2=0x7faf9c1143e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 shutdown_connections 2024-04-20T21:38:38.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:38.843+0000 7fafa0c00640 1 -- 172.21.15.7:0/992785051 wait complete. 2024-04-20T21:38:38.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:38:38.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:38:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.109+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1672845533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101700 msgr2=0x7f4ac8105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.109+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1672845533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101700 0x7f4ac8105bc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ab401a910 tx=0x7f4ab403cd00 comp rx=0 tx=0).stop 2024-04-20T21:38:39.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.110+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1672845533 shutdown_connections 2024-04-20T21:38:39.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.110+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1672845533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac8101700 0x7f4ac8105bc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.110+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1672845533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac81061e0 0x7f4ac8106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.111+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1672845533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 0x7f4ac81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.111+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1672845533 >> 172.21.15.7:0/1672845533 conn(0x7f4ac80fc7f0 msgr2=0x7f4ac80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:39.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.112+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1672845533 shutdown_connections 2024-04-20T21:38:39.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.112+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1672845533 wait complete. 2024-04-20T21:38:39.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.115+0000 7f4ac7e00640 1 Processor -- start 2024-04-20T21:38:39.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.116+0000 7f4ac7e00640 1 -- start start 2024-04-20T21:38:39.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.117+0000 7f4ac7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 0x7f4ac81150d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.117+0000 7f4ac7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 0x7f4ac8111160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.117+0000 7f4ac7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 0x7f4ac81116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 0x7f4ac8111160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 0x7f4ac8111160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40928/0 (socket says 172.21.15.7:40928) 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac6000640 1 -- 172.21.15.7:0/1456361406 learned_addr learned my addr 172.21.15.7:0/1456361406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac6a00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 0x7f4ac81150d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac7400640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 0x7f4ac81116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8109af0 con 0x7f4ac8100de0 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8109970 con 0x7f4ac81061e0 2024-04-20T21:38:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.118+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8109c70 con 0x7f4ac8101700 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.120+0000 7f4ac7400640 1 -- 172.21.15.7:0/1456361406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 msgr2=0x7f4ac8111160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.120+0000 7f4ac7400640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 0x7f4ac8111160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.120+0000 7f4ac7400640 1 -- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 msgr2=0x7f4ac81150d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.120+0000 7f4ac7400640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 0x7f4ac81150d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.120+0000 7f4ac7400640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac8111da0 con 0x7f4ac81061e0 2024-04-20T21:38:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.121+0000 7f4ac7400640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 0x7f4ac81116a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000cab0 tx=0x7f4ac000cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.122+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000ae20 con 0x7f4ac81061e0 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.122+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac000f930 con 0x7f4ac81061e0 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.122+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac81a2bf0 con 0x7f4ac81061e0 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.122+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0018b30 con 0x7f4ac81061e0 2024-04-20T21:38:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.122+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81a2ff0 con 0x7f4ac81061e0 2024-04-20T21:38:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.127+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac8069b40 con 0x7f4ac81061e0 2024-04-20T21:38:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.130+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4ac0018cd0 con 0x7f4ac81061e0 2024-04-20T21:38:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.135+0000 7f4abf400640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 0x7f4aac0a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.135+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4ac00d0a80 con 0x7f4ac81061e0 2024-04-20T21:38:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.138+0000 7f4ac6a00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 0x7f4aac0a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.138+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f4ac0014030 con 0x7f4ac81061e0 2024-04-20T21:38:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.140+0000 7f4ac6a00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 0x7f4aac0a62d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009890 tx=0x7f4ab8008040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.667+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ac81a32e0 con 0x7f4ac81061e0 2024-04-20T21:38:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.669+0000 7f4abf400640 1 -- 172.21.15.7:0/1456361406 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f4ac009d110 con 0x7f4ac81061e0 2024-04-20T21:38:39.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:38:39.671 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:38:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:38:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 msgr2=0x7f4aac0a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 0x7f4aac0a62d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009890 tx=0x7f4ab8008040 comp rx=0 tx=0).stop 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 msgr2=0x7f4ac81116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 0x7f4ac81116a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000cab0 tx=0x7f4ac000cf70 comp rx=0 tx=0).stop 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 shutdown_connections 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4aac0a3e10 0x7f4aac0a62d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac81061e0 0x7f4ac81116a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4ac8101700 0x7f4ac8111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 --2- 172.21.15.7:0/1456361406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac8100de0 0x7f4ac81150d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.673+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 >> 172.21.15.7:0/1456361406 conn(0x7f4ac80fc7f0 msgr2=0x7f4ac8103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.674+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 shutdown_connections 2024-04-20T21:38:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:39.676+0000 7f4ac7e00640 1 -- 172.21.15.7:0/1456361406 wait complete. 2024-04-20T21:38:39.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:38:39.830 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:39 smithi007 ceph-mon[16867]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 8.67k op/s 2024-04-20T21:38:39.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:39 smithi016 ceph-mon[21409]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 8.67k op/s 2024-04-20T21:38:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:39 smithi186 ceph-mon[22157]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 49 MiB/s wr, 8.67k op/s 2024-04-20T21:38:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:40 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1456361406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:40 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1456361406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:41.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:40 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1456361406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:38:41.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:41 smithi016 ceph-mon[21409]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 5.89k op/s 2024-04-20T21:38:41.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:41 smithi186 ceph-mon[22157]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 5.89k op/s 2024-04-20T21:38:42.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:41 smithi007 ceph-mon[16867]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 5.89k op/s 2024-04-20T21:38:42.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:42 smithi016 ceph-mon[21409]: pgmap v164: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 5.02k op/s 2024-04-20T21:38:42.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:42 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:42.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:42 smithi186 ceph-mon[22157]: pgmap v164: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 5.02k op/s 2024-04-20T21:38:42.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:42 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:43.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:42 smithi007 ceph-mon[16867]: pgmap v164: 97 pgs: 97 active+clean; 21 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 5.02k op/s 2024-04-20T21:38:43.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:42 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:38:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:43 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:43 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:43.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:43 smithi016 ceph-mon[21409]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 7.36k op/s 2024-04-20T21:38:43.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:43 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:38:43.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:43 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:43.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:43 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:43.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:43 smithi186 ceph-mon[22157]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 7.36k op/s 2024-04-20T21:38:43.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:43 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:38:44.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:43 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:38:44.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:43 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:38:44.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:43 smithi007 ceph-mon[16867]: pgmap v165: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 7.36k op/s 2024-04-20T21:38:44.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:43 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:38:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 -- 172.21.15.7:0/499916045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48104d60 msgr2=0x7f5c48108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 --2- 172.21.15.7:0/499916045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48104d60 0x7f5c48108da0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002bc30 comp rx=0 tx=0).stop 2024-04-20T21:38:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 -- 172.21.15.7:0/499916045 shutdown_connections 2024-04-20T21:38:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 --2- 172.21.15.7:0/499916045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48104d60 0x7f5c48108da0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 --2- 172.21.15.7:0/499916045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c48109480 0x7f5c48111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 --2- 172.21.15.7:0/499916045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c481043b0 0x7f5c48104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 -- 172.21.15.7:0/499916045 >> 172.21.15.7:0/499916045 conn(0x7f5c480ffbf0 msgr2=0x7f5c48102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 -- 172.21.15.7:0/499916045 shutdown_connections 2024-04-20T21:38:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.523+0000 7f5c4d400640 1 -- 172.21.15.7:0/499916045 wait complete. 2024-04-20T21:38:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.524+0000 7f5c4d400640 1 Processor -- start 2024-04-20T21:38:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.524+0000 7f5c4d400640 1 -- start start 2024-04-20T21:38:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 0x7f5c4819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 0x7f5c4819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c4819f450 0x7f5c4819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c48114560 con 0x7f5c481043b0 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481143e0 con 0x7f5c48109480 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c4d400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481146e0 con 0x7f5c4819f450 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 0x7f5c4819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 0x7f5c4819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 0x7f5c4819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42474/0 (socket says 172.21.15.7:42474) 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c46800640 1 -- 172.21.15.7:0/3214968589 learned_addr learned my addr 172.21.15.7:0/3214968589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.525+0000 7f5c47200640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c4819f450 0x7f5c4819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 -- 172.21.15.7:0/3214968589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c4819f450 msgr2=0x7f5c4819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c4819f450 0x7f5c4819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 -- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 msgr2=0x7f5c4819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 0x7f5c4819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 -- 172.21.15.7:0/3214968589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4819ff40 con 0x7f5c48109480 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c45e00640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 0x7f5c4819b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c30036040 tx=0x7f5c30005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c3f400640 1 -- 172.21.15.7:0/3214968589 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30043020 con 0x7f5c48109480 2024-04-20T21:38:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c3f400640 1 -- 172.21.15.7:0/3214968589 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c30005000 con 0x7f5c48109480 2024-04-20T21:38:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.526+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c481a01d0 con 0x7f5c48109480 2024-04-20T21:38:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.527+0000 7f5c3f400640 1 -- 172.21.15.7:0/3214968589 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30003af0 con 0x7f5c48109480 2024-04-20T21:38:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.527+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c481a79f0 con 0x7f5c48109480 2024-04-20T21:38:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.530+0000 7f5c3f400640 1 -- 172.21.15.7:0/3214968589 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5c30046020 con 0x7f5c48109480 2024-04-20T21:38:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.530+0000 7f5c50178740 1 Processor -- start 2024-04-20T21:38:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 -- start start 2024-04-20T21:38:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8026ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c80270e0 0x5584c802f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 0x5584c80320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584c7f83050 con 0x5584c80267c0 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584c7f29490 con 0x5584c802fcc0 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.531+0000 7f5c50178740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5584c7f7f010 con 0x5584c80270e0 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 0x5584c80320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c47200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8026ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c47200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8026ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42486/0 (socket says 172.21.15.7:42486) 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c47200640 1 -- 172.21.15.7:0/2743736221 learned_addr learned my addr 172.21.15.7:0/2743736221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c46800640 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c80270e0 0x5584c802f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 -- 172.21.15.7:0/2743736221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c80270e0 msgr2=0x5584c802f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c80270e0 0x5584c802f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 -- 172.21.15.7:0/2743736221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 msgr2=0x5584c8026ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8026ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c45e00640 1 -- 172.21.15.7:0/2743736221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c7f31070 con 0x5584c802fcc0 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.532+0000 7f5c3f400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 0x7f5c2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c47200640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 0x7f5c2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c3f400640 1 -- 172.21.15.7:0/3214968589 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5c300f25c0 con 0x7f5c48109480 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 0x5584c80320b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c300efe10 tx=0x7f5c300f8580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fc83f71c864619 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c37400640 1 -- 172.21.15.7:0/2743736221 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300be430 con 0x5584c802fcc0 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c37400640 1 -- 172.21.15.7:0/2743736221 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300bed60 con 0x5584c802fcc0 2024-04-20T21:38:44.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.533+0000 7f5c37400640 1 -- 172.21.15.7:0/2743736221 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300bdfc0 con 0x5584c802fcc0 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 -- 172.21.15.7:0/2743736221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 msgr2=0x5584c80320b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 0x5584c80320b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5c300efe10 tx=0x7f5c300f8580 comp rx=0 tx=0).stop 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c47200640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 0x7f5c2c0a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009e60 tx=0x7f5c40005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 -- 172.21.15.7:0/2743736221 shutdown_connections 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c802fcc0 0x5584c80320b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c80270e0 0x5584c802f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 --2- 172.21.15.7:0/2743736221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8026ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 -- 172.21.15.7:0/2743736221 >> 172.21.15.7:0/2743736221 conn(0x5584c8020ef0 msgr2=0x5584c80212d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 -- 172.21.15.7:0/2743736221 shutdown_connections 2024-04-20T21:38:44.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.534+0000 7f5c50178740 1 -- 172.21.15.7:0/2743736221 wait complete. 2024-04-20T21:38:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.535+0000 7f5c50178740 1 -- start start 2024-04-20T21:38:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.535+0000 7f5c50178740 1 Processor -- start 2024-04-20T21:38:44.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c802d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c802d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584c8032870 con 0x5584c80267c0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584c7f3c860 con 0x5584c80270e0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5584c80329f0 con 0x5584c802fcc0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c50178740 10 client.0 _ll_register_callbacks cb 0x5584c7f0afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c47200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c802d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42488/0 (socket says 172.21.15.7:42488) 2024-04-20T21:38:44.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c802d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c46800640 1 -- 172.21.15.7:0/1521306162 learned_addr learned my addr 172.21.15.7:0/1521306162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.536+0000 7f5c45e00640 1 -- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 msgr2=0x5584c802d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c45e00640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c802d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c45e00640 1 -- 172.21.15.7:0/1521306162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 msgr2=0x5584c802cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c45e00640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c45e00640 1 -- 172.21.15.7:0/1521306162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c7f40950 con 0x5584c80270e0 2024-04-20T21:38:44.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c47200640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c802d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c45e00640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c802d1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c300bc910 tx=0x7f5c300be7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c182786a3358639f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300be1a0 con 0x5584c80270e0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5c30003c90 con 0x5584c80270e0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c300bed60 con 0x5584c80270e0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c300dc050 con 0x5584c80270e0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c50178740 10 client.26207 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.537+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584c7f0e820 con 0x5584c80270e0 2024-04-20T21:38:44.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300bed60 con 0x5584c80270e0 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f5c300f8cc0 con 0x5584c80270e0 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 10 client.26207 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 10 client.26207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 20 client.26207 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c803f7d0 0x5584c8041bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 4 client.26207 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.538+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c7f90a70 con 0x5584c803f7d0 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.539+0000 7f5c46800640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c803f7d0 0x5584c8041bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.539+0000 7f5c46800640 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c803f7d0 0x5584c8041bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ec2ebad82f446a2 server_cookie=ba85367695632e44 in_seq=0 out_seq=0 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.539+0000 7f5c2be00640 10 client.26207 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c2be00640 1 -- 172.21.15.7:0/1521306162 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584c7f90a70 con 0x5584c803f7d0 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c2be00640 10 client.26207 handle_command_reply: tid=0 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 client.26207 shutdown 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 msgr2=0x5584c802d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c802d1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c300bc910 tx=0x7f5c300be7e0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 shutdown_connections 2024-04-20T21:38:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c803f7d0 0x5584c8041bc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c802d1a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c802d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c2be00640 0 client.26207 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.540+0000 7f5c50178740 1 --2- 172.21.15.7:0/1521306162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.541+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 >> 172.21.15.7:0/1521306162 conn(0x5584c8020ef0 msgr2=0x5584c8022c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.541+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 shutdown_connections 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.541+0000 7f5c50178740 1 -- 172.21.15.7:0/1521306162 wait complete. 2024-04-20T21:38:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.541+0000 7f5c50178740 20 client.26207 trim_cache size 0 max 16384 2024-04-20T21:38:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.556+0000 7f5c50178740 1 Processor -- start 2024-04-20T21:38:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.556+0000 7f5c50178740 1 -- start start 2024-04-20T21:38:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8047380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ae110 con 0x5584c80267c0 2024-04-20T21:38:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ab9d0 con 0x5584c80270e0 2024-04-20T21:38:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c50178740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481a9820 con 0x5584c802fcc0 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38872/0 (socket says 172.21.15.7:38872) 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c45e00640 1 -- 172.21.15.7:0/2971346149 learned_addr learned my addr 172.21.15.7:0/2971346149 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.557+0000 7f5c47200640 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8047380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 -- 172.21.15.7:0/2971346149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 msgr2=0x5584c8047380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8047380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 -- 172.21.15.7:0/2971346149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 msgr2=0x5584c802f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 -- 172.21.15.7:0/2971346149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c8041f30 con 0x5584c80270e0 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c30006010 tx=0x7f5c300f8580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e58508f76c325f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c36a00640 1 -- 172.21.15.7:0/2971346149 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300be1a0 con 0x5584c80270e0 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c36a00640 1 -- 172.21.15.7:0/2971346149 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300bed60 con 0x5584c80270e0 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.558+0000 7f5c36a00640 1 -- 172.21.15.7:0/2971346149 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300bdfc0 con 0x5584c80270e0 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 -- 172.21.15.7:0/2971346149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 msgr2=0x5584c803b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c30006010 tx=0x7f5c300f8580 comp rx=0 tx=0).stop 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 -- 172.21.15.7:0/2971346149 shutdown_connections 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c803b640 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8047380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 --2- 172.21.15.7:0/2971346149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c802f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 -- 172.21.15.7:0/2971346149 >> 172.21.15.7:0/2971346149 conn(0x5584c8020ef0 msgr2=0x5584c8028040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 -- 172.21.15.7:0/2971346149 shutdown_connections 2024-04-20T21:38:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.559+0000 7f5c50178740 1 -- 172.21.15.7:0/2971346149 wait complete. 2024-04-20T21:38:44.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.560+0000 7f5c50178740 1 -- start start 2024-04-20T21:38:44.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.560+0000 7f5c50178740 1 Processor -- start 2024-04-20T21:38:44.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.560+0000 7f5c50178740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8037de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8033690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481a9e90 con 0x5584c80267c0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481ab300 con 0x5584c80270e0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5c481aa560 con 0x5584c802fcc0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c50178740 10 client.0 _ll_register_callbacks cb 0x7f5c300f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8037de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38880/0 (socket says 172.21.15.7:38880) 2024-04-20T21:38:44.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 -- 172.21.15.7:0/2102126547 learned_addr learned my addr 172.21.15.7:0/2102126547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c47200640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8033690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 -- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 msgr2=0x5584c8033690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8033690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 -- 172.21.15.7:0/2102126547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 msgr2=0x5584c8037de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8037de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c45e00640 1 -- 172.21.15.7:0/2102126547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c7c953f0 con 0x5584c80270e0 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c47200640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8033690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:44.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.561+0000 7f5c46800640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8037de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c45e00640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c300bb030 tx=0x7f5c300f02c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b9ed3fb8275eec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c2be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300be430 con 0x5584c80270e0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5c300be8c0 con 0x5584c80270e0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c300bed60 con 0x5584c80270e0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5c300bdfc0 con 0x5584c80270e0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c50178740 10 client.26213 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584c7d5f7d0 con 0x5584c80270e0 2024-04-20T21:38:44.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.562+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c300bed60 con 0x5584c80270e0 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f5c300c6ed0 con 0x5584c80270e0 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 10 client.26213 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 10 client.26213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 20 client.26213 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c80a5a20 0x5584c80a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 4 client.26213 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584c7f8a8e0 con 0x5584c80a5a20 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.563+0000 7f5c46800640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c80a5a20 0x5584c80a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c46800640 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c80a5a20 0x5584c80a7e40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18a2db02f52632ca server_cookie=4a068b043f088daf in_seq=0 out_seq=0 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c2be00640 10 client.26213 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c2be00640 1 -- 172.21.15.7:0/2102126547 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584c7f8a8e0 con 0x5584c80a5a20 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c2be00640 10 client.26213 handle_command_reply: tid=0 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 client.26213 shutdown 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 msgr2=0x5584c8033150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c300bb030 tx=0x7f5c300f02c0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 shutdown_connections 2024-04-20T21:38:44.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5584c80a5a20 0x5584c80a7e40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c80270e0 0x5584c8033150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c802fcc0 0x5584c8033690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 --2- 172.21.15.7:0/2102126547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c80267c0 0x5584c8037de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c2be00640 0 client.26213 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 >> 172.21.15.7:0/2102126547 conn(0x5584c8020ef0 msgr2=0x5584c8016eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.564+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 shutdown_connections 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c50178740 1 -- 172.21.15.7:0/2102126547 wait complete. 2024-04-20T21:38:44.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c50178740 20 client.26213 trim_cache size 0 max 16384 2024-04-20T21:38:44.572 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:38:44.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 msgr2=0x7f5c2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 0x7f5c2c0a6260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009e60 tx=0x7f5c40005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 msgr2=0x7f5c4819b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.565+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 0x7f5c4819b0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5c30036040 tx=0x7f5c30005e80 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 shutdown_connections 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5c2c0a3da0 0x7f5c2c0a6260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c48109480 0x7f5c4819b0a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5c4819f450 0x7f5c4819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 --2- 172.21.15.7:0/3214968589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c481043b0 0x7f5c4819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 >> 172.21.15.7:0/3214968589 conn(0x7f5c480ffbf0 msgr2=0x7f5c48101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 shutdown_connections 2024-04-20T21:38:44.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.566+0000 7f5c4d400640 1 -- 172.21.15.7:0/3214968589 wait complete. 2024-04-20T21:38:44.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:38:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 -- 172.21.15.7:0/1849455366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102f20 msgr2=0x7f863c103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 --2- 172.21.15.7:0/1849455366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102f20 0x7f863c103300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8630009860 tx=0x7f863002bc00 comp rx=0 tx=0).stop 2024-04-20T21:38:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 -- 172.21.15.7:0/1849455366 shutdown_connections 2024-04-20T21:38:44.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 --2- 172.21.15.7:0/1849455366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c10c340 0x7f863c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 --2- 172.21.15.7:0/1849455366 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 --2- 172.21.15.7:0/1849455366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c102f20 0x7f863c103300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 -- 172.21.15.7:0/1849455366 >> 172.21.15.7:0/1849455366 conn(0x7f863c0fc760 msgr2=0x7f863c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 -- 172.21.15.7:0/1849455366 shutdown_connections 2024-04-20T21:38:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.806+0000 7f8641e00640 1 -- 172.21.15.7:0/1849455366 wait complete. 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.807+0000 7f8641e00640 1 Processor -- start 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.807+0000 7f8641e00640 1 -- start start 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10c340 0x7f863c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f863c112100 con 0x7f863c102f20 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c111f80 con 0x7f863c10c340 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f863c112280 con 0x7f863c103840 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8640a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8640a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42538/0 (socket says 172.21.15.7:42538) 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8640a00640 1 -- 172.21.15.7:0/382533783 learned_addr learned my addr 172.21.15.7:0/382533783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f863be00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.808+0000 7f8641400640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10c340 0x7f863c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 -- 172.21.15.7:0/382533783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 msgr2=0x7f863c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 -- 172.21.15.7:0/382533783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10c340 msgr2=0x7f863c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10c340 0x7f863c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 -- 172.21.15.7:0/382533783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c19d680 con 0x7f863c102f20 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f863be00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8640a00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8630037040 tx=0x7f86300344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8639600640 1 -- 172.21.15.7:0/382533783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630041020 con 0x7f863c102f20 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c19d910 con 0x7f863c102f20 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.809+0000 7f8639600640 1 -- 172.21.15.7:0/382533783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863002ca80 con 0x7f863c102f20 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.810+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1a5200 con 0x7f863c102f20 2024-04-20T21:38:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.810+0000 7f8639600640 1 -- 172.21.15.7:0/382533783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630039040 con 0x7f863c102f20 2024-04-20T21:38:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.813+0000 7f86461f1740 1 Processor -- start 2024-04-20T21:38:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.813+0000 7f86461f1740 1 -- start start 2024-04-20T21:38:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.813+0000 7f8639600640 1 -- 172.21.15.7:0/382533783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8630044020 con 0x7f863c102f20 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.813+0000 7f86461f1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f86461f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f86461f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x56088948fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f86461f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5608893e5050 con 0x560889484ea0 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f86461f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56088938b490 con 0x56088948da80 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f86461f1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5608893e1010 con 0x560889484580 2024-04-20T21:38:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50590/0 (socket says 172.21.15.7:50590) 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 -- 172.21.15.7:0/832628662 learned_addr learned my addr 172.21.15.7:0/832628662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8640a00640 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f863be00640 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x56088948fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 -- 172.21.15.7:0/832628662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 msgr2=0x56088948fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x56088948fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 -- 172.21.15.7:0/832628662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 msgr2=0x56088948d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.814+0000 7f8641400640 1 -- 172.21.15.7:0/832628662 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560889393070 con 0x560889484580 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.815+0000 7f8641400640 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f863400e480 tx=0x7f86340095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b98dd46b36ba97c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.815+0000 7f862dc00640 1 -- 172.21.15.7:0/832628662 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634003970 con 0x560889484580 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.815+0000 7f862dc00640 1 -- 172.21.15.7:0/832628662 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863400d7d0 con 0x560889484580 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.815+0000 7f862dc00640 1 -- 172.21.15.7:0/832628662 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004400 con 0x560889484580 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 -- 172.21.15.7:0/832628662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 msgr2=0x560889484960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f863400e480 tx=0x7f86340095c0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 -- 172.21.15.7:0/832628662 shutdown_connections 2024-04-20T21:38:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f8639600640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 0x7f86240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x56088948fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 --2- 172.21.15.7:0/832628662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889484960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f86461f1740 1 -- 172.21.15.7:0/832628662 >> 172.21.15.7:0/832628662 conn(0x56088947ecb0 msgr2=0x56088947f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.816+0000 7f8639600640 1 -- 172.21.15.7:0/382533783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f86300f1e10 con 0x7f863c102f20 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f86461f1740 1 -- 172.21.15.7:0/832628662 shutdown_connections 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f8641400640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 0x7f86240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f86461f1740 1 -- 172.21.15.7:0/832628662 wait complete. 2024-04-20T21:38:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f86461f1740 1 -- start start 2024-04-20T21:38:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f86461f1740 1 Processor -- start 2024-04-20T21:38:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.817+0000 7f8641400640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 0x7f86240a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e40 tx=0x7f8634009b10 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x56088948b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560889490870 con 0x560889484ea0 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56088939e860 con 0x560889484580 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5608894909f0 con 0x56088948da80 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f86461f1740 10 client.0 _ll_register_callbacks cb 0x56088936cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f8641400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x56088948b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f8640a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f8640a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38918/0 (socket says 172.21.15.7:38918) 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f8640a00640 1 -- 172.21.15.7:0/2426854452 learned_addr learned my addr 172.21.15.7:0/2426854452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.818+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42556/0 (socket says 172.21.15.7:42556) 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 -- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 msgr2=0x56088948b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x56088948b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 -- 172.21.15.7:0/2426854452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 msgr2=0x56088948b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 -- 172.21.15.7:0/2426854452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608893a2950 con 0x560889484580 2024-04-20T21:38:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f863be00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8640a00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f863002c210 tx=0x7f8630009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74117147fa04561d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8622000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300bd810 con 0x560889484580 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f86300be110 con 0x560889484580 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.819+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f863400d7d0 con 0x560889484580 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86300ef9d0 con 0x560889484580 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 10 client.26225 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560889370820 con 0x560889484580 2024-04-20T21:38:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f863400d7d0 con 0x560889484580 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f863003c030 con 0x560889484580 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 10 client.26225 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 10 client.26225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 20 client.26225 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56088949dab0 0x56088949ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 4 client.26225 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.820+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608893f2a70 con 0x56088949dab0 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.821+0000 7f863be00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56088949dab0 0x56088949ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.821+0000 7f863be00640 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56088949dab0 0x56088949ff70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ddc9546e228d895 server_cookie=baa2f461d62b96f6 in_seq=0 out_seq=0 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.821+0000 7f8622000640 10 client.26225 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.822+0000 7f8622000640 1 -- 172.21.15.7:0/2426854452 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608893f2a70 con 0x56088949dab0 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.822+0000 7f8622000640 10 client.26225 handle_command_reply: tid=0 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.822+0000 7f86461f1740 1 client.26225 shutdown 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.822+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 msgr2=0x56088948aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.822+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f863002c210 tx=0x7f8630009490 comp rx=0 tx=0).stop 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 shutdown_connections 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56088949dab0 0x56088949ff70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x56088948b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 --2- 172.21.15.7:0/2426854452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x56088948aac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 >> 172.21.15.7:0/2426854452 conn(0x56088947ecb0 msgr2=0x560889480a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f8622000640 0 client.26225 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 shutdown_connections 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 1 -- 172.21.15.7:0/2426854452 wait complete. 2024-04-20T21:38:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.823+0000 7f86461f1740 20 client.26225 trim_cache size 0 max 16384 2024-04-20T21:38:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 Processor -- start 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 -- start start 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x5608894a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ae090 con 0x560889484ea0 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ab950 con 0x56088948da80 2024-04-20T21:38:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.838+0000 7f86461f1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1a97a0 con 0x560889484580 2024-04-20T21:38:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8640a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8641400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x5608894a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8640a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50608/0 (socket says 172.21.15.7:50608) 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42564/0 (socket says 172.21.15.7:42564) 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8640a00640 1 -- 172.21.15.7:0/1744298451 learned_addr learned my addr 172.21.15.7:0/1744298451 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 -- 172.21.15.7:0/1744298451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 msgr2=0x560889499730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 -- 172.21.15.7:0/1744298451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 msgr2=0x5608894a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x5608894a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f863be00640 1 -- 172.21.15.7:0/1744298451 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608890f73f0 con 0x560889484ea0 2024-04-20T21:38:44.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8640a00640 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.839+0000 7f8641400640 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x5608894a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f863be00640 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5608894a07f0 tx=0x7f86280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b044eca7b908e0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f862d200640 1 -- 172.21.15.7:0/1744298451 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862800fc80 con 0x560889484ea0 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f862d200640 1 -- 172.21.15.7:0/1744298451 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862800fe20 con 0x560889484ea0 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f862d200640 1 -- 172.21.15.7:0/1744298451 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628022170 con 0x560889484ea0 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f86461f1740 1 -- 172.21.15.7:0/1744298451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 msgr2=0x56088948b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.840+0000 7f86461f1740 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5608894a07f0 tx=0x7f86280046c0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 -- 172.21.15.7:0/1744298451 shutdown_connections 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56088948da80 0x5608894a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x56088948b190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 --2- 172.21.15.7:0/1744298451 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560889484580 0x560889499730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 -- 172.21.15.7:0/1744298451 >> 172.21.15.7:0/1744298451 conn(0x56088947ecb0 msgr2=0x56088948aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 -- 172.21.15.7:0/1744298451 shutdown_connections 2024-04-20T21:38:44.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 -- 172.21.15.7:0/1744298451 wait complete. 2024-04-20T21:38:44.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 -- start start 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.841+0000 7f86461f1740 1 Processor -- start 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x560889487e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x560889488340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x5608895030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1a9e10 con 0x560889484ea0 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1ab280 con 0x560889484580 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f863c1aa4e0 con 0x56088948da80 2024-04-20T21:38:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f86461f1740 10 client.0 _ll_register_callbacks cb 0x7f8630009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x560889488340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f863be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x560889488340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42566/0 (socket says 172.21.15.7:42566) 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f8640a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x560889487e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f8641400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x5608895030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.842+0000 7f863be00640 1 -- 172.21.15.7:0/3732482376 learned_addr learned my addr 172.21.15.7:0/3732482376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 -- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 msgr2=0x5608895030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x5608895030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 -- 172.21.15.7:0/3732482376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 msgr2=0x560889488340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x560889488340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 -- 172.21.15.7:0/3732482376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608891c17d0 con 0x560889484580 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8640a00640 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x560889487e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86300ba830 tx=0x7f8630005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7109aaf59f48c7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8622000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86300db060 con 0x560889484580 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862800fe20 con 0x560889484580 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.843+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f86300bd510 con 0x560889484580 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86300ba150 con 0x560889484580 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 10 client.26231 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5608891c17d0 con 0x560889484580 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862800fe20 con 0x560889484580 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f86300d70f0 con 0x560889484580 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 10 client.26231 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 10 client.26231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 20 client.26231 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560889504a80 0x560889506ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 4 client.26231 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.844+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608893ec8e0 con 0x560889504a80 2024-04-20T21:38:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f863be00640 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560889504a80 0x560889506ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f863be00640 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560889504a80 0x560889506ea0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ac811ba1ee9f9b9 server_cookie=b245babf36ba7c43 in_seq=0 out_seq=0 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f8622000640 10 client.26231 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f8622000640 1 -- 172.21.15.7:0/3732482376 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608893ec8e0 con 0x560889504a80 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f8622000640 10 client.26231 handle_command_reply: tid=0 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.845+0000 7f86461f1740 1 client.26231 shutdown 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 msgr2=0x560889487e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x560889487e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86300ba830 tx=0x7f8630005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 shutdown_connections 2024-04-20T21:38:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x560889504a80 0x560889506ea0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56088948da80 0x5608895030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560889484ea0 0x560889488340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f8622000640 0 client.26231 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 --2- 172.21.15.7:0/3732482376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560889484580 0x560889487e00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 >> 172.21.15.7:0/3732482376 conn(0x56088947ecb0 msgr2=0x56088948b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 shutdown_connections 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 1 -- 172.21.15.7:0/3732482376 wait complete. 2024-04-20T21:38:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.846+0000 7f86461f1740 20 client.26231 trim_cache size 0 max 16384 2024-04-20T21:38:44.853 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:38:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 msgr2=0x7f86240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 0x7f86240a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e40 tx=0x7f8634009b10 comp rx=0 tx=0).stop 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 msgr2=0x7f863c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8630037040 tx=0x7f86300344c0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 shutdown_connections 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f86240a3d20 0x7f86240a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f863c10c340 0x7f863c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f863c103840 0x7f863c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 --2- 172.21.15.7:0/382533783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863c102f20 0x7f863c1986e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 >> 172.21.15.7:0/382533783 conn(0x7f863c0fc760 msgr2=0x7f863c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 shutdown_connections 2024-04-20T21:38:44.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:44.847+0000 7f8641e00640 1 -- 172.21.15.7:0/382533783 wait complete. 2024-04-20T21:38:44.861 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:44 smithi007 ceph-mon[16867]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-20T21:38:44.861 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:44 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:38:44.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:38:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:44 smithi016 ceph-mon[21409]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-20T21:38:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:44 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:38:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:44 smithi186 ceph-mon[22157]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-20T21:38:44.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:44 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:38:45.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.071+0000 7fc000e00640 1 -- 172.21.15.7:0/493073136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 msgr2=0x7fbffc104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.071+0000 7fc000e00640 1 --2- 172.21.15.7:0/493073136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc104590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02bc40 comp rx=0 tx=0).stop 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 -- 172.21.15.7:0/493073136 shutdown_connections 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 --2- 172.21.15.7:0/493073136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc104b60 0x7fbffc108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 --2- 172.21.15.7:0/493073136 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc109280 0x7fbffc110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 --2- 172.21.15.7:0/493073136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc104590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 -- 172.21.15.7:0/493073136 >> 172.21.15.7:0/493073136 conn(0x7fbffc0ffa30 msgr2=0x7fbffc101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 -- 172.21.15.7:0/493073136 shutdown_connections 2024-04-20T21:38:45.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.072+0000 7fc000e00640 1 -- 172.21.15.7:0/493073136 wait complete. 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.073+0000 7fc000e00640 1 Processor -- start 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.073+0000 7fc000e00640 1 -- start start 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 0x7fbffc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114310 con 0x7fbffc1041b0 2024-04-20T21:38:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114190 con 0x7fbffc109280 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fc000e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc114490 con 0x7fbffc19a6f0 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 0x7fbffc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 0x7fbffc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50640/0 (socket says 172.21.15.7:50640) 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffbe00640 1 -- 172.21.15.7:0/2704462537 learned_addr learned my addr 172.21.15.7:0/2704462537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38956/0 (socket says 172.21.15.7:38956) 2024-04-20T21:38:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 -- 172.21.15.7:0/2704462537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 msgr2=0x7fbffc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 0x7fbffc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 -- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 msgr2=0x7fbffc1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.074+0000 7fbffaa00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fbffaa00640 1 -- 172.21.15.7:0/2704462537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc19b2f0 con 0x7fbffc109280 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fbffaa00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9c0 tx=0x7fbfe800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fbff3e00640 1 -- 172.21.15.7:0/2704462537 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800cd80 con 0x7fbffc109280 2024-04-20T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fbff3e00640 1 -- 172.21.15.7:0/2704462537 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe80035f0 con 0x7fbffc109280 2024-04-20T21:38:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fbff3e00640 1 -- 172.21.15.7:0/2704462537 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8010690 con 0x7fbffc109280 2024-04-20T21:38:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc1a73d0 con 0x7fbffc109280 2024-04-20T21:38:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.075+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a7780 con 0x7fbffc109280 2024-04-20T21:38:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 Processor -- start 2024-04-20T21:38:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 -- start start 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935a90 0x5557d993e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d993e670 0x5557d9940a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5557d989a050 con 0x5557d9935a90 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5557d9840490 con 0x5557d9935170 2024-04-20T21:38:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.080+0000 7fc00501c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5557d9896010 con 0x5557d993e670 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935a90 0x5557d993e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38972/0 (socket says 172.21.15.7:38972) 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 -- 172.21.15.7:0/2246842576 learned_addr learned my addr 172.21.15.7:0/2246842576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbff3e00640 1 -- 172.21.15.7:0/2704462537 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbfe80037a0 con 0x7fbffc109280 2024-04-20T21:38:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 -- 172.21.15.7:0/2246842576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d993e670 msgr2=0x5557d9940a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d993e670 0x5557d9940a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 -- 172.21.15.7:0/2246842576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935a90 msgr2=0x5557d993e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935a90 0x5557d993e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 -- 172.21.15.7:0/2246842576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d9848070 con 0x5557d9935170 2024-04-20T21:38:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.081+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbff4011490 tx=0x7fbff4011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40a3387c35549c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.082+0000 7fbff0c00640 1 -- 172.21.15.7:0/2246842576 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff401e070 con 0x5557d9935170 2024-04-20T21:38:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.082+0000 7fbff0c00640 1 -- 172.21.15.7:0/2246842576 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff400fae0 con 0x5557d9935170 2024-04-20T21:38:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.082+0000 7fbff0c00640 1 -- 172.21.15.7:0/2246842576 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4019ba0 con 0x5557d9935170 2024-04-20T21:38:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.082+0000 7fbff3e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 0x7fbfe00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.082+0000 7fbff3e00640 1 -- 172.21.15.7:0/2704462537 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfe80d0a10 con 0x7fbffc109280 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 -- 172.21.15.7:0/2246842576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 msgr2=0x5557d9935550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbff4011490 tx=0x7fbff4011950 comp rx=0 tx=0).stop 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fbffb400640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 0x7fbfe00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 -- 172.21.15.7:0/2246842576 shutdown_connections 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d993e670 0x5557d9940a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935a90 0x5557d993e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 --2- 172.21.15.7:0/2246842576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9935550 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 -- 172.21.15.7:0/2246842576 >> 172.21.15.7:0/2246842576 conn(0x5557d992f8a0 msgr2=0x5557d992fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 -- 172.21.15.7:0/2246842576 shutdown_connections 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.083+0000 7fc00501c740 1 -- 172.21.15.7:0/2246842576 wait complete. 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 -- start start 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fbffb400640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 0x7fbfe00a6260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006010 tx=0x7fbfec02e040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 Processor -- start 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935170 0x5557d993b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 0x5557d993bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5557d9941460 con 0x5557d993e670 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5557d9853860 con 0x5557d9935a90 2024-04-20T21:38:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5557d99415e0 con 0x5557d9935170 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.084+0000 7fc00501c740 10 client.0 _ll_register_callbacks cb 0x5557d9821fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 0x5557d993bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffb400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935170 0x5557d993b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42604/0 (socket says 172.21.15.7:42604) 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 -- 172.21.15.7:0/3557995022 learned_addr learned my addr 172.21.15.7:0/3557995022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 -- 172.21.15.7:0/3557995022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935170 msgr2=0x5557d993b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935170 0x5557d993b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 -- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 msgr2=0x5557d993bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 0x5557d993bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 -- 172.21.15.7:0/3557995022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d9857950 con 0x5557d993e670 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffaa00640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 0x5557d993bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbffbe00640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005c40 tx=0x7fbff4019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9f1264ec68d575 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.085+0000 7fbfe5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff401e040 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbff400f700 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff400fae0 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbff4032050 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fc00501c740 10 client.16050 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557d9825820 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.086+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff400fae0 con 0x5557d993e670 2024-04-20T21:38:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fbff4021030 con 0x5557d993e670 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 10 client.16050 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 20 client.16050 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d994e6c0 0x5557d9950ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 4 client.16050 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557d98a7a70 con 0x5557d994e6c0 2024-04-20T21:38:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.087+0000 7fbffb400640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d994e6c0 0x5557d9950ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.088+0000 7fbffb400640 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d994e6c0 0x5557d9950ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39311c2da99e6d43 server_cookie=a27c329285ca49b5 in_seq=0 out_seq=0 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.088+0000 7fbfe5600640 10 client.16050 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fbfe5600640 1 -- 172.21.15.7:0/3557995022 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557d98a7a70 con 0x5557d994e6c0 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fbfe5600640 10 client.16050 handle_command_reply: tid=0 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 client.16050 shutdown 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 msgr2=0x5557d993c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005c40 tx=0x7fbff4019640 comp rx=0 tx=0).stop 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 shutdown_connections 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935a90 0x5557d993bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d994e6c0 0x5557d9950ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d993c1e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 --2- 172.21.15.7:0/3557995022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935170 0x5557d993b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fbfe5600640 0 client.16050 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 >> 172.21.15.7:0/3557995022 conn(0x5557d992f8a0 msgr2=0x5557d99315f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 shutdown_connections 2024-04-20T21:38:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 1 -- 172.21.15.7:0/3557995022 wait complete. 2024-04-20T21:38:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.089+0000 7fc00501c740 20 client.16050 trim_cache size 0 max 16384 2024-04-20T21:38:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.104+0000 7fc00501c740 1 Processor -- start 2024-04-20T21:38:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.104+0000 7fc00501c740 1 -- start start 2024-04-20T21:38:45.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 0x5557d993de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d994a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1b2550 con 0x5557d9935170 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1afe10 con 0x5557d993e670 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fc00501c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1adc80 con 0x5557d9935a90 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 0x5557d993de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38986/0 (socket says 172.21.15.7:38986) 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 -- 172.21.15.7:0/2145096805 learned_addr learned my addr 172.21.15.7:0/2145096805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 0x5557d993de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42608/0 (socket says 172.21.15.7:42608) 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffaa00640 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d994a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 -- 172.21.15.7:0/2145096805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 msgr2=0x5557d994a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d994a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 -- 172.21.15.7:0/2145096805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 msgr2=0x5557d993de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 0x5557d993de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.105+0000 7fbffbe00640 1 -- 172.21.15.7:0/2145096805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d95ac3f0 con 0x5557d993e670 2024-04-20T21:38:45.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fbffbe00640 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbff4011990 tx=0x7fbff4009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc467771bb975d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fbfe7e00640 1 -- 172.21.15.7:0/2145096805 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff401e070 con 0x5557d993e670 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fbfe7e00640 1 -- 172.21.15.7:0/2145096805 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff4029070 con 0x5557d993e670 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fbfe7e00640 1 -- 172.21.15.7:0/2145096805 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff400f600 con 0x5557d993e670 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fc00501c740 1 -- 172.21.15.7:0/2145096805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 msgr2=0x5557d9956220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.106+0000 7fc00501c740 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbff4011990 tx=0x7fbff4009230 comp rx=0 tx=0).stop 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 -- 172.21.15.7:0/2145096805 shutdown_connections 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d993e670 0x5557d9956220 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d994a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 --2- 172.21.15.7:0/2145096805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d9935170 0x5557d993de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 -- 172.21.15.7:0/2145096805 >> 172.21.15.7:0/2145096805 conn(0x5557d992f8a0 msgr2=0x5557d9936ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 -- 172.21.15.7:0/2145096805 shutdown_connections 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 -- 172.21.15.7:0/2145096805 wait complete. 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 -- start start 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.107+0000 7fc00501c740 1 Processor -- start 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9946750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d9946c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ae2d0 con 0x5557d993e670 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1af740 con 0x5557d9935170 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ae9a0 con 0x5557d9935a90 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fc00501c740 10 client.0 _ll_register_callbacks cb 0x7fbff40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fbffbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42618/0 (socket says 172.21.15.7:42618) 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fbffbe00640 1 -- 172.21.15.7:0/111991276 learned_addr learned my addr 172.21.15.7:0/111991276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:45.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fbffb400640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9946750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.108+0000 7fbffaa00640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d9946c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 -- 172.21.15.7:0/111991276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 msgr2=0x5557d9946c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d9946c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 -- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 msgr2=0x5557d9946750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9946750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 -- 172.21.15.7:0/111991276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d96767d0 con 0x5557d993e670 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbffbe00640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbff4010000 tx=0x7fbff4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef4a08300dcd3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbfe5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:38:45.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff401e070 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbff4004a00 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff4029070 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbff4044050 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fc00501c740 10 client.16056 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.109+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557d99b4430 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff4029070 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fbff4021040 con 0x5557d993e670 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 10 client.16056 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 20 client.16056 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d99b4a30 0x5557d99b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 4 client.16056 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557d98a18e0 con 0x5557d99b4a30 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.110+0000 7fbffb400640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d99b4a30 0x5557d99b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fbffb400640 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d99b4a30 0x5557d99b6e20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5edd894e494e4419 server_cookie=819fce7abedcaf6a in_seq=0 out_seq=0 2024-04-20T21:38:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fbfe5600640 10 client.16056 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fbfe5600640 1 -- 172.21.15.7:0/111991276 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557d98a18e0 con 0x5557d99b4a30 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fbfe5600640 10 client.16056 handle_command_reply: tid=0 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 client.16056 shutdown 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 msgr2=0x5557d9941c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbff4010000 tx=0x7fbff4009ca0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 shutdown_connections 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5557d99b4a30 0x5557d99b6e20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5557d993e670 0x5557d9941c40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5557d9935a90 0x5557d9946c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 --2- 172.21.15.7:0/111991276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5557d9935170 0x5557d9946750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fbfe5600640 0 client.16056 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.111+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 >> 172.21.15.7:0/111991276 conn(0x5557d992f8a0 msgr2=0x5557d992e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 shutdown_connections 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc00501c740 1 -- 172.21.15.7:0/111991276 wait complete. 2024-04-20T21:38:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc00501c740 20 client.16056 trim_cache size 0 max 16384 2024-04-20T21:38:45.122 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:38:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 msgr2=0x7fbfe00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 0x7fbfe00a6260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfec006010 tx=0x7fbfec02e040 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 msgr2=0x7fbffc1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800e9c0 tx=0x7fbfe800ee80 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 shutdown_connections 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbfe00a3da0 0x7fbfe00a6260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbffc109280 0x7fbffc1a0b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbffc19a6f0 0x7fbffc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 --2- 172.21.15.7:0/2704462537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbffc1041b0 0x7fbffc1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.112+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 >> 172.21.15.7:0/2704462537 conn(0x7fbffc0ffa30 msgr2=0x7fbffc100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.113+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 shutdown_connections 2024-04-20T21:38:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:45.113+0000 7fc000e00640 1 -- 172.21.15.7:0/2704462537 wait complete. 2024-04-20T21:38:45.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:45 smithi016 ceph-mon[21409]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 7.44k op/s 2024-04-20T21:38:45.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:45 smithi186 ceph-mon[22157]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 7.44k op/s 2024-04-20T21:38:46.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:45 smithi007 ceph-mon[16867]: pgmap v166: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 43 MiB/s wr, 7.44k op/s 2024-04-20T21:38:46.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:38:46.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:38:46.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:38:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.519+0000 7ff6c2600640 1 -- 172.21.15.7:0/2956467372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 msgr2=0x7ff6bc0746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.519+0000 7ff6c2600640 1 --2- 172.21.15.7:0/2956467372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc0746f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009ca0 tx=0x7ff6b402c190 comp rx=0 tx=0).stop 2024-04-20T21:38:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.520+0000 7ff6c2600640 1 -- 172.21.15.7:0/2956467372 shutdown_connections 2024-04-20T21:38:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.520+0000 7ff6c2600640 1 --2- 172.21.15.7:0/2956467372 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.520+0000 7ff6c2600640 1 --2- 172.21.15.7:0/2956467372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc074c30 0x7ff6bc10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.520+0000 7ff6c2600640 1 --2- 172.21.15.7:0/2956467372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc0746f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.520+0000 7ff6c2600640 1 -- 172.21.15.7:0/2956467372 >> 172.21.15.7:0/2956467372 conn(0x7ff6bc0fda10 msgr2=0x7ff6bc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.521+0000 7ff6c2600640 1 -- 172.21.15.7:0/2956467372 shutdown_connections 2024-04-20T21:38:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.521+0000 7ff6c2600640 1 -- 172.21.15.7:0/2956467372 wait complete. 2024-04-20T21:38:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.523+0000 7ff6c2600640 1 Processor -- start 2024-04-20T21:38:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 -- start start 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc074c30 0x7ff6bc10b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc10ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc114610 con 0x7ff6bc10e7e0 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc114490 con 0x7ff6bc074310 2024-04-20T21:38:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.524+0000 7ff6c2600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc114790 con 0x7ff6bc074c30 2024-04-20T21:38:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39020/0 (socket says 172.21.15.7:39020) 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 -- 172.21.15.7:0/3610680877 learned_addr learned my addr 172.21.15.7:0/3610680877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6baa00640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc074c30 0x7ff6bc10b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc10ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 -- 172.21.15.7:0/3610680877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc074c30 msgr2=0x7ff6bc10b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc074c30 0x7ff6bc10b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.525+0000 7ff6bb400640 1 -- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 msgr2=0x7ff6bc10ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.526+0000 7ff6bb400640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc10ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.526+0000 7ff6bb400640 1 -- 172.21.15.7:0/3610680877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1ab350 con 0x7ff6bc074310 2024-04-20T21:38:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.526+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6bb400640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6b402c7a0 tx=0x7ff6b4003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6b3e00640 1 -- 172.21.15.7:0/3610680877 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4039040 con 0x7ff6bc074310 2024-04-20T21:38:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6b3e00640 1 -- 172.21.15.7:0/3610680877 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6b4044070 con 0x7ff6bc074310 2024-04-20T21:38:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6b3e00640 1 -- 172.21.15.7:0/3610680877 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4032a70 con 0x7ff6bc074310 2024-04-20T21:38:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1ab520 con 0x7ff6bc074310 2024-04-20T21:38:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.527+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc1ab9f0 con 0x7ff6bc074310 2024-04-20T21:38:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.538+0000 7ff6c5427740 1 Processor -- start 2024-04-20T21:38:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.538+0000 7ff6c5427740 1 -- start start 2024-04-20T21:38:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.538+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d149e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6c5427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558d1493fd20 con 0x558d149db820 2024-04-20T21:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6c5427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d148e63e0 con 0x558d149daf00 2024-04-20T21:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6c5427740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558d1493bce0 con 0x558d149e4400 2024-04-20T21:38:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39024/0 (socket says 172.21.15.7:39024) 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.539+0000 7ff6bbe00640 1 -- 172.21.15.7:0/4028991711 learned_addr learned my addr 172.21.15.7:0/4028991711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6bbe00640 1 -- 172.21.15.7:0/4028991711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 msgr2=0x558d149e67f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d149e67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6bbe00640 1 -- 172.21.15.7:0/4028991711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 msgr2=0x558d149e3ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:38:46.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149e3ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6bbe00640 1 -- 172.21.15.7:0/4028991711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d14905980 con 0x558d149daf00 2024-04-20T21:38:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.540+0000 7ff6b3e00640 1 -- 172.21.15.7:0/3610680877 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff6b4032c10 con 0x7ff6bc074310 2024-04-20T21:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.542+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a990 tx=0x7ff6ac00ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f359c5354a7855 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.542+0000 7ff6b0c00640 1 -- 172.21.15.7:0/4028991711 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac015070 con 0x558d149daf00 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.542+0000 7ff6b0c00640 1 -- 172.21.15.7:0/4028991711 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac00cbe0 con 0x558d149daf00 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.542+0000 7ff6b0c00640 1 -- 172.21.15.7:0/4028991711 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0185c0 con 0x558d149daf00 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.543+0000 7ff6b3e00640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 0x7ff6a00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.543+0000 7ff6c5427740 1 -- 172.21.15.7:0/4028991711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 msgr2=0x558d149db2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.543+0000 7ff6c5427740 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a990 tx=0x7ff6ac00ae50 comp rx=0 tx=0).stop 2024-04-20T21:38:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.543+0000 7ff6b3e00640 1 -- 172.21.15.7:0/3610680877 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff6b40f2ce0 con 0x7ff6bc074310 2024-04-20T21:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6c5427740 1 -- 172.21.15.7:0/4028991711 shutdown_connections 2024-04-20T21:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6c5427740 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d149e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6c5427740 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149e3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6c5427740 1 --2- 172.21.15.7:0/4028991711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149db2e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6c5427740 1 -- 172.21.15.7:0/4028991711 >> 172.21.15.7:0/4028991711 conn(0x558d149d5630 msgr2=0x558d149d5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.544+0000 7ff6bb400640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 0x7ff6a00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.545+0000 7ff6c5427740 1 -- 172.21.15.7:0/4028991711 shutdown_connections 2024-04-20T21:38:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.545+0000 7ff6c5427740 1 -- 172.21.15.7:0/4028991711 wait complete. 2024-04-20T21:38:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.546+0000 7ff6c5427740 1 -- start start 2024-04-20T21:38:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.546+0000 7ff6c5427740 1 Processor -- start 2024-04-20T21:38:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.546+0000 7ff6bb400640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 0x7ff6a00a6320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x558d149da9d0 tx=0x7ff6b402c190 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.547+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149db820 0x558d149e19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558d149e71f0 con 0x558d149e4400 2024-04-20T21:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558d148f97a0 con 0x558d149db820 2024-04-20T21:38:46.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x558d149e7370 con 0x558d149daf00 2024-04-20T21:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50698/0 (socket says 172.21.15.7:50698) 2024-04-20T21:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6bbe00640 1 -- 172.21.15.7:0/1495997722 learned_addr learned my addr 172.21.15.7:0/1495997722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.548+0000 7ff6c5427740 10 client.0 _ll_register_callbacks cb 0x558d148fbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6baa00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149db820 0x558d149e19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 -- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149db820 msgr2=0x558d149e19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149db820 0x558d149e19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 -- 172.21.15.7:0/1495997722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 msgr2=0x558d149e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149e1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.549+0000 7ff6bbe00640 1 -- 172.21.15.7:0/1495997722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1471a7e0 con 0x558d149daf00 2024-04-20T21:38:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.550+0000 7ff6a5600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:38:46.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.550+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a960 tx=0x7ff6ac006c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93676000020422be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.551+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac022070 con 0x558d149daf00 2024-04-20T21:38:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.551+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac00cbe0 con 0x558d149daf00 2024-04-20T21:38:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.551+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff6ac00c9e0 con 0x558d149daf00 2024-04-20T21:38:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.553+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac022070 con 0x558d149daf00 2024-04-20T21:38:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.553+0000 7ff6c5427740 10 client.26227 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.553+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1477af70 con 0x558d149daf00 2024-04-20T21:38:46.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.553+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac00cbe0 con 0x558d149daf00 2024-04-20T21:38:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.554+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ff6ac02f5e0 con 0x558d149daf00 2024-04-20T21:38:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.554+0000 7ff6c5427740 10 client.26227 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.554+0000 7ff6c5427740 10 client.26227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.554+0000 7ff6c5427740 20 client.26227 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.554+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d149f47b0 0x558d149f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.555+0000 7ff6c5427740 4 client.26227 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.555+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1494d740 con 0x558d149f47b0 2024-04-20T21:38:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.555+0000 7ff6baa00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d149f47b0 0x558d149f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.556+0000 7ff6baa00640 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d149f47b0 0x558d149f6ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223017b7fe0aa98e server_cookie=3d368d2f1c04e052 in_seq=0 out_seq=0 2024-04-20T21:38:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.557+0000 7ff6a5600640 10 client.26227 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.558+0000 7ff6a5600640 1 -- 172.21.15.7:0/1495997722 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d1494d740 con 0x558d149f47b0 2024-04-20T21:38:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.558+0000 7ff6a5600640 10 client.26227 handle_command_reply: tid=0 2024-04-20T21:38:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.558+0000 7ff6c5427740 1 client.26227 shutdown 2024-04-20T21:38:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.559+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 msgr2=0x558d149e1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.559+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00a960 tx=0x7ff6ac006c70 comp rx=0 tx=0).stop 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 shutdown_connections 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149db820 0x558d149e19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d149f47b0 0x558d149f6ba0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 --2- 172.21.15.7:0/1495997722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149daf00 0x558d149e1490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6a5600640 0 client.26227 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:46.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.561+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 >> 172.21.15.7:0/1495997722 conn(0x558d149d5630 msgr2=0x558d149d7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.562+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 shutdown_connections 2024-04-20T21:38:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.562+0000 7ff6c5427740 1 -- 172.21.15.7:0/1495997722 wait complete. 2024-04-20T21:38:46.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.563+0000 7ff6c5427740 20 client.26227 trim_cache size 0 max 16384 2024-04-20T21:38:46.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.616+0000 7ff6c5427740 1 Processor -- start 2024-04-20T21:38:46.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.616+0000 7ff6c5427740 1 -- start start 2024-04-20T21:38:46.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149e3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6c5427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b6c10 con 0x558d149e4400 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6c5427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b44d0 con 0x558d149daf00 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.618+0000 7ff6c5427740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b2320 con 0x558d149db820 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.617+0000 7ff6baa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.618+0000 7ff6baa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50712/0 (socket says 172.21.15.7:50712) 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.618+0000 7ff6baa00640 1 -- 172.21.15.7:0/860638253 learned_addr learned my addr 172.21.15.7:0/860638253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.618+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149e3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.619+0000 7ff6baa00640 1 -- 172.21.15.7:0/860638253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 msgr2=0x558d149e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.619+0000 7ff6baa00640 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149e3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.619+0000 7ff6baa00640 1 -- 172.21.15.7:0/860638253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 msgr2=0x558d149fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.619+0000 7ff6baa00640 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.620+0000 7ff6baa00640 1 -- 172.21.15.7:0/860638253 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d14612170 con 0x558d149db820 2024-04-20T21:38:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.620+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.621+0000 7ff6baa00640 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x558d149e0d10 tx=0x7ff6a8009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523f8731488cc3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.622+0000 7ff6a7e00640 1 -- 172.21.15.7:0/860638253 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8009e50 con 0x558d149db820 2024-04-20T21:38:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.622+0000 7ff6a7e00640 1 -- 172.21.15.7:0/860638253 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6a801f030 con 0x558d149db820 2024-04-20T21:38:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.622+0000 7ff6a7e00640 1 -- 172.21.15.7:0/860638253 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8014850 con 0x558d149db820 2024-04-20T21:38:46.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 -- 172.21.15.7:0/860638253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 msgr2=0x558d149f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x558d149e0d10 tx=0x7ff6a8009110 comp rx=0 tx=0).stop 2024-04-20T21:38:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 -- 172.21.15.7:0/860638253 shutdown_connections 2024-04-20T21:38:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149e4400 0x558d149fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149db820 0x558d149f00e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 --2- 172.21.15.7:0/860638253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d149e3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.623+0000 7ff6c5427740 1 -- 172.21.15.7:0/860638253 >> 172.21.15.7:0/860638253 conn(0x558d149d5630 msgr2=0x558d149d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.625+0000 7ff6c5427740 1 -- 172.21.15.7:0/860638253 shutdown_connections 2024-04-20T21:38:46.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.626+0000 7ff6c5427740 1 -- 172.21.15.7:0/860638253 wait complete. 2024-04-20T21:38:46.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.626+0000 7ff6c5427740 1 -- start start 2024-04-20T21:38:46.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.627+0000 7ff6c5427740 1 Processor -- start 2024-04-20T21:38:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d14a564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d14a53880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b2990 con 0x558d149db820 2024-04-20T21:38:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b3e00 con 0x558d149daf00 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc1b3060 con 0x558d149e4400 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.629+0000 7ff6c5427740 10 client.0 _ll_register_callbacks cb 0x7ff6ac00a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.630+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d14a53880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.630+0000 7ff6bbe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d14a53880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50728/0 (socket says 172.21.15.7:50728) 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.630+0000 7ff6bbe00640 1 -- 172.21.15.7:0/3861625235 learned_addr learned my addr 172.21.15.7:0/3861625235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.630+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d14a564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.632+0000 7ff6baa00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.633+0000 7ff6bbe00640 1 -- 172.21.15.7:0/3861625235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 msgr2=0x558d14a53880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.633+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d14a53880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.633+0000 7ff6bbe00640 1 -- 172.21.15.7:0/3861625235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 msgr2=0x558d149ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.633+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.633+0000 7ff6bbe00640 1 -- 172.21.15.7:0/3861625235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d14a53dc0 con 0x558d149daf00 2024-04-20T21:38:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.634+0000 7ff6bbe00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d14a564a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c680 tx=0x7ff6ac01d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00b0ae009c48684 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.634+0000 7ff6a5600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.634+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac034030 con 0x558d149daf00 2024-04-20T21:38:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.634+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff6ac02f380 con 0x558d149daf00 2024-04-20T21:38:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.635+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6a801f030 con 0x558d149daf00 2024-04-20T21:38:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.636+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac03e050 con 0x558d149daf00 2024-04-20T21:38:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.636+0000 7ff6c5427740 10 client.26264 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.636+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d14a6c0e0 con 0x558d149daf00 2024-04-20T21:38:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6ac02f580 con 0x558d149daf00 2024-04-20T21:38:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ff6ac027460 con 0x558d149daf00 2024-04-20T21:38:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 10 client.26264 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 20 client.26264 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d14a6c740 0x558d14a6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 4 client.26264 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:38:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.638+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff6a801f030 con 0x558d14a6c740 2024-04-20T21:38:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.641+0000 7ff6baa00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d14a6c740 0x558d14a6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.642+0000 7ff6baa00640 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d14a6c740 0x558d14a6eb60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fac857170640b3 server_cookie=560684a533c4672e in_seq=0 out_seq=0 2024-04-20T21:38:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.642+0000 7ff6a5600640 10 client.26264 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.644+0000 7ff6a5600640 1 -- 172.21.15.7:0/3861625235 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff6a801f030 con 0x558d14a6c740 2024-04-20T21:38:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.646+0000 7ff6a5600640 10 client.26264 handle_command_reply: tid=0 2024-04-20T21:38:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.646+0000 7ff6c5427740 1 client.26264 shutdown 2024-04-20T21:38:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.646+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 msgr2=0x558d14a564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.646+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d14a564a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c680 tx=0x7ff6ac01d040 comp rx=0 tx=0).stop 2024-04-20T21:38:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 shutdown_connections 2024-04-20T21:38:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x558d14a6c740 0x558d14a6eb60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558d149e4400 0x558d14a53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d149db820 0x558d149ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 --2- 172.21.15.7:0/3861625235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558d149daf00 0x558d14a564a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 >> 172.21.15.7:0/3861625235 conn(0x558d149d5630 msgr2=0x558d149d6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6a5600640 0 client.26264 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 shutdown_connections 2024-04-20T21:38:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 1 -- 172.21.15.7:0/3861625235 wait complete. 2024-04-20T21:38:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.647+0000 7ff6c5427740 20 client.26264 trim_cache size 0 max 16384 2024-04-20T21:38:46.649 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:38:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:38:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "c33ed40b-394a-4727-98ed-452aa2716278", 2024-04-20T21:38:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:38:46.649 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:38:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.648+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 msgr2=0x7ff6a00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.648+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 0x7ff6a00a6320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x558d149da9d0 tx=0x7ff6b402c190 comp rx=0 tx=0).stop 2024-04-20T21:38:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 msgr2=0x7ff6bc10e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff6b402c7a0 tx=0x7ff6b4003ba0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 shutdown_connections 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6bc10e7e0 0x7ff6bc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff6bc074c30 0x7ff6bc10b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff6a00a3e60 0x7ff6a00a6320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 --2- 172.21.15.7:0/3610680877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6bc074310 0x7ff6bc10e260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.649+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 >> 172.21.15.7:0/3610680877 conn(0x7ff6bc0fda10 msgr2=0x7ff6bc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.650+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 shutdown_connections 2024-04-20T21:38:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.650+0000 7ff6c2600640 1 -- 172.21.15.7:0/3610680877 wait complete. 2024-04-20T21:38:46.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:38:46.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 -- 172.21.15.7:0/1730404839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc80102b40 msgr2=0x7efc80106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 --2- 172.21.15.7:0/1730404839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc80102b40 0x7efc80106b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009860 tx=0x7efc6c02bb90 comp rx=0 tx=0).stop 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 -- 172.21.15.7:0/1730404839 shutdown_connections 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 --2- 172.21.15.7:0/1730404839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80107260 0x7efc8010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 --2- 172.21.15.7:0/1730404839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc80102b40 0x7efc80106b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 --2- 172.21.15.7:0/1730404839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80102190 0x7efc80102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:46.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.997+0000 7efc85c00640 1 -- 172.21.15.7:0/1730404839 >> 172.21.15.7:0/1730404839 conn(0x7efc800fda10 msgr2=0x7efc800ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:46.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.998+0000 7efc85c00640 1 -- 172.21.15.7:0/1730404839 shutdown_connections 2024-04-20T21:38:46.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:46.998+0000 7efc85c00640 1 -- 172.21.15.7:0/1730404839 wait complete. 2024-04-20T21:38:46.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.000+0000 7efc85c00640 1 Processor -- start 2024-04-20T21:38:47.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.001+0000 7efc85c00640 1 -- start start 2024-04-20T21:38:47.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.001+0000 7efc85c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 0x7efc8010e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.001+0000 7efc85c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.002+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 0x7efc8010e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.002+0000 7efc7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.002+0000 7efc7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39066/0 (socket says 172.21.15.7:39066) 2024-04-20T21:38:47.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.002+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 0x7efc8010e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42682/0 (socket says 172.21.15.7:42682) 2024-04-20T21:38:47.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.001+0000 7efc85c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc801a3810 0x7efc801a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.004+0000 7efc7e000640 1 -- 172.21.15.7:0/691414762 learned_addr learned my addr 172.21.15.7:0/691414762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.004+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc80112320 con 0x7efc80102190 2024-04-20T21:38:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.004+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc801121a0 con 0x7efc80107260 2024-04-20T21:38:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.004+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efc801124a0 con 0x7efc801a3810 2024-04-20T21:38:47.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.005+0000 7efc7e000640 1 -- 172.21.15.7:0/691414762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc801a3810 msgr2=0x7efc801a3cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:38:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.005+0000 7efc7e000640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc801a3810 0x7efc801a3cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.005+0000 7efc7e000640 1 -- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 msgr2=0x7efc8010e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.005+0000 7efc7e000640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 0x7efc8010e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.005+0000 7efc7e000640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc801a4300 con 0x7efc80107260 2024-04-20T21:38:47.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.006+0000 7efc7e000640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc6c037040 tx=0x7efc6c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.007+0000 7efc77400640 1 -- 172.21.15.7:0/691414762 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c041020 con 0x7efc80107260 2024-04-20T21:38:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.007+0000 7efc77400640 1 -- 172.21.15.7:0/691414762 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c02ca10 con 0x7efc80107260 2024-04-20T21:38:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.007+0000 7efc77400640 1 -- 172.21.15.7:0/691414762 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c039040 con 0x7efc80107260 2024-04-20T21:38:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.007+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc801a4590 con 0x7efc80107260 2024-04-20T21:38:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.007+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801abce0 con 0x7efc80107260 2024-04-20T21:38:47.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.017+0000 7efc8885e740 1 Processor -- start 2024-04-20T21:38:47.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.019+0000 7efc77400640 1 -- 172.21.15.7:0/691414762 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7efc6c044020 con 0x7efc80107260 2024-04-20T21:38:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 -- start start 2024-04-20T21:38:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b28a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b293160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55907b1f0150 con 0x55907b2936a0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55907b196370 con 0x55907b28aac0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.020+0000 7efc8885e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55907b1ec110 con 0x55907b28a1a0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.021+0000 7efc7e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.021+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b293160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.021+0000 7efc7e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42686/0 (socket says 172.21.15.7:42686) 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.021+0000 7efc7e000640 1 -- 172.21.15.7:0/4236166440 learned_addr learned my addr 172.21.15.7:0/4236166440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.021+0000 7efc7f400640 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b28a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 -- 172.21.15.7:0/4236166440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 msgr2=0x55907b28a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b28a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 -- 172.21.15.7:0/4236166440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 msgr2=0x55907b293160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b293160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 -- 172.21.15.7:0/4236166440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907afcc7d0 con 0x55907b2936a0 2024-04-20T21:38:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.022+0000 7efc7e000640 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0ef690 tx=0x7efc6c0f5f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21576bb15f6159a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.025+0000 7efc77400640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 0x7efc640a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.025+0000 7efc77400640 1 -- 172.21.15.7:0/691414762 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc6c0f1f00 con 0x7efc80107260 2024-04-20T21:38:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.026+0000 7efc6be00640 1 -- 172.21.15.7:0/4236166440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0ba240 con 0x55907b2936a0 2024-04-20T21:38:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.026+0000 7efc6be00640 1 -- 172.21.15.7:0/4236166440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c0ff070 con 0x55907b2936a0 2024-04-20T21:38:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.026+0000 7efc6be00640 1 -- 172.21.15.7:0/4236166440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0bd900 con 0x55907b2936a0 2024-04-20T21:38:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.027+0000 7efc7f400640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 0x7efc640a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.029+0000 7efc8885e740 1 -- 172.21.15.7:0/4236166440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 msgr2=0x55907b295a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.029+0000 7efc8885e740 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0ef690 tx=0x7efc6c0f5f10 comp rx=0 tx=0).stop 2024-04-20T21:38:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.030+0000 7efc8885e740 1 -- 172.21.15.7:0/4236166440 shutdown_connections 2024-04-20T21:38:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.030+0000 7efc8885e740 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b293160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.030+0000 7efc8885e740 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b295a90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.030+0000 7efc8885e740 1 --2- 172.21.15.7:0/4236166440 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b28a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.030+0000 7efc8885e740 1 -- 172.21.15.7:0/4236166440 >> 172.21.15.7:0/4236166440 conn(0x55907b2848d0 msgr2=0x55907b284cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:47.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.031+0000 7efc7f400640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 0x7efc640a6360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc78005fd0 tx=0x7efc78005ea0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.031+0000 7efc8885e740 1 -- 172.21.15.7:0/4236166440 shutdown_connections 2024-04-20T21:38:47.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.031+0000 7efc8885e740 1 -- 172.21.15.7:0/4236166440 wait complete. 2024-04-20T21:38:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.032+0000 7efc8885e740 1 -- start start 2024-04-20T21:38:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.033+0000 7efc8885e740 1 Processor -- start 2024-04-20T21:38:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.034+0000 7efc8885e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.034+0000 7efc8885e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28aac0 0x55907b290c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39084/0 (socket says 172.21.15.7:39084) 2024-04-20T21:38:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc8885e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 0x55907b2911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc8885e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55907b2a7260 con 0x55907b28aac0 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc8885e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55907b1a9740 con 0x55907b28a1a0 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc8885e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55907b2a73e0 con 0x55907b2936a0 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc8885e740 10 client.0 _ll_register_callbacks cb 0x55907b1a4b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.035+0000 7efc7ea00640 1 -- 172.21.15.7:0/1456641890 learned_addr learned my addr 172.21.15.7:0/1456641890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7e000640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28aac0 0x55907b290c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7f400640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 0x55907b2911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 -- 172.21.15.7:0/1456641890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 msgr2=0x55907b2911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 0x55907b2911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 -- 172.21.15.7:0/1456641890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28aac0 msgr2=0x55907b290c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28aac0 0x55907b290c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 -- 172.21.15.7:0/1456641890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55907af023f0 con 0x55907b28a1a0 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7f400640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 0x55907b2911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc7ea00640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc70009fd0 tx=0x7efc70012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5c8309bdcc7019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.036+0000 7efc68c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.037+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70004020 con 0x55907b28a1a0 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.037+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc700041c0 con 0x55907b28a1a0 2024-04-20T21:38:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.037+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc6c0ff070 con 0x55907b28a1a0 2024-04-20T21:38:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.038+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc700043c0 con 0x55907b28a1a0 2024-04-20T21:38:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.038+0000 7efc8885e740 10 client.26276 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.038+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55907aec20f0 con 0x55907b28a1a0 2024-04-20T21:38:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.039+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c0ff070 con 0x55907b28a1a0 2024-04-20T21:38:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.039+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7efc70003c70 con 0x55907b28a1a0 2024-04-20T21:38:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.040+0000 7efc8885e740 10 client.26276 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.040+0000 7efc8885e740 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.040+0000 7efc8885e740 20 client.26276 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.040+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b2b4610 0x55907b2b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.041+0000 7efc8885e740 4 client.26276 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:38:47.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.041+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55907b1fdb70 con 0x55907b2b4610 2024-04-20T21:38:47.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.041+0000 7efc7e000640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b2b4610 0x55907b2b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.042+0000 7efc7e000640 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b2b4610 0x55907b2b6a30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e54aa4df7c66f8 server_cookie=b163d0f05ac83a2b in_seq=0 out_seq=0 2024-04-20T21:38:47.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.042+0000 7efc68c00640 10 client.26276 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:47.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc68c00640 1 -- 172.21.15.7:0/1456641890 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55907b1fdb70 con 0x55907b2b4610 2024-04-20T21:38:47.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc68c00640 10 client.26276 handle_command_reply: tid=0 2024-04-20T21:38:47.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc8885e740 1 client.26276 shutdown 2024-04-20T21:38:47.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 msgr2=0x55907b290740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efc70009fd0 tx=0x7efc70012b60 comp rx=0 tx=0).stop 2024-04-20T21:38:47.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 shutdown_connections 2024-04-20T21:38:47.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.044+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b2b4610 0x55907b2b6a30 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b2936a0 0x55907b2911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28aac0 0x55907b290c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 --2- 172.21.15.7:0/1456641890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28a1a0 0x55907b290740 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 >> 172.21.15.7:0/1456641890 conn(0x55907b2848d0 msgr2=0x55907b286620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc68c00640 0 client.26276 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:47.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 shutdown_connections 2024-04-20T21:38:47.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.045+0000 7efc8885e740 1 -- 172.21.15.7:0/1456641890 wait complete. 2024-04-20T21:38:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.047+0000 7efc8885e740 20 client.26276 trim_cache size 0 max 16384 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.094+0000 7efc8885e740 1 Processor -- start 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 -- start start 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28a1a0 0x55907b2b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b2936a0 0x55907b2bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc801ae3d0 con 0x55907b28a1a0 2024-04-20T21:38:47.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc80073e40 con 0x55907b2936a0 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.095+0000 7efc8885e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efc80071cb0 con 0x55907b28aac0 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50770/0 (socket says 172.21.15.7:50770) 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 -- 172.21.15.7:0/2897849198 learned_addr learned my addr 172.21.15.7:0/2897849198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7f400640 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b2936a0 0x55907b2bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28a1a0 0x55907b2b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 -- 172.21.15.7:0/2897849198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b2936a0 msgr2=0x55907b2bc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b2936a0 0x55907b2bc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 -- 172.21.15.7:0/2897849198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28a1a0 msgr2=0x55907b2b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28a1a0 0x55907b2b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.096+0000 7efc7e000640 1 -- 172.21.15.7:0/2897849198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55907b28a8f0 con 0x55907b28aac0 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.097+0000 7efc7e000640 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0bdf30 tx=0x7efc6c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a342f232734c8ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.097+0000 7efc6b400640 1 -- 172.21.15.7:0/2897849198 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0d7070 con 0x55907b28aac0 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.097+0000 7efc6b400640 1 -- 172.21.15.7:0/2897849198 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c0ff040 con 0x55907b28aac0 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.097+0000 7efc6b400640 1 -- 172.21.15.7:0/2897849198 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0c6e60 con 0x55907b28aac0 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 -- 172.21.15.7:0/2897849198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 msgr2=0x55907b2bc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0bdf30 tx=0x7efc6c005f90 comp rx=0 tx=0).stop 2024-04-20T21:38:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 -- 172.21.15.7:0/2897849198 shutdown_connections 2024-04-20T21:38:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b2936a0 0x55907b2bc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28aac0 0x55907b2bc300 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 --2- 172.21.15.7:0/2897849198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b28a1a0 0x55907b2b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.099+0000 7efc8885e740 1 -- 172.21.15.7:0/2897849198 >> 172.21.15.7:0/2897849198 conn(0x55907b2848d0 msgr2=0x55907b2838c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.100+0000 7efc8885e740 1 -- 172.21.15.7:0/2897849198 shutdown_connections 2024-04-20T21:38:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.100+0000 7efc8885e740 1 -- 172.21.15.7:0/2897849198 wait complete. 2024-04-20T21:38:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.100+0000 7efc8885e740 1 -- start start 2024-04-20T21:38:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.101+0000 7efc8885e740 1 Processor -- start 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.102+0000 7efc8885e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b2a82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.102+0000 7efc8885e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.102+0000 7efc8885e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b29d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.103+0000 7efc7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.103+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b2a82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.103+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b2a82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50772/0 (socket says 172.21.15.7:50772) 2024-04-20T21:38:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.103+0000 7efc7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39110/0 (socket says 172.21.15.7:39110) 2024-04-20T21:38:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc8885e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc80072300 con 0x55907b2936a0 2024-04-20T21:38:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc8885e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efc80073770 con 0x55907b28aac0 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc8885e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7efc800729d0 con 0x55907b28a1a0 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc8885e740 10 client.0 _ll_register_callbacks cb 0x7efc70009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7ea00640 1 -- 172.21.15.7:0/749330332 learned_addr learned my addr 172.21.15.7:0/749330332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7f400640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b29d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7e000640 1 -- 172.21.15.7:0/749330332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 msgr2=0x55907b2a82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7e000640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b2a82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7e000640 1 -- 172.21.15.7:0/749330332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 msgr2=0x55907b29d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.104+0000 7efc7e000640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b29d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.105+0000 7efc7e000640 1 -- 172.21.15.7:0/749330332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55907b2a90b0 con 0x55907b28aac0 2024-04-20T21:38:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.105+0000 7efc7f400640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b29d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:38:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.106+0000 7efc7e000640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55907b289e20 tx=0x7efc6c02c0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59606ba02d28b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:38:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.106+0000 7efc68c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:38:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.106+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0d7040 con 0x55907b28aac0 2024-04-20T21:38:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.106+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc6c02c530 con 0x55907b28aac0 2024-04-20T21:38:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.106+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc6c0ff040 con 0x55907b28aac0 2024-04-20T21:38:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.108+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efc6c0c6e60 con 0x55907b28aac0 2024-04-20T21:38:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.108+0000 7efc8885e740 10 client.26282 fetch_fsmap learned FSMap version 27 2024-04-20T21:38:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.108+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55907b29e6c0 con 0x55907b28aac0 2024-04-20T21:38:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c0ff040 con 0x55907b28aac0 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7efc6c0f3030 con 0x55907b28aac0 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 10 client.26282 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 20 client.26282 populate_metadata read hostname 'smithi007' 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b29e950 0x55907b2a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:38:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 4 client.26282 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:38:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.113+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55907b1f79e0 con 0x55907b29e950 2024-04-20T21:38:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.116+0000 7efc7ea00640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b29e950 0x55907b2a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:38:47.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc7ea00640 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b29e950 0x55907b2a0d40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7baf604a7f2240a server_cookie=c2d14e326b8788c6 in_seq=0 out_seq=0 2024-04-20T21:38:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc68c00640 10 client.26282 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc68c00640 1 -- 172.21.15.7:0/749330332 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55907b1f79e0 con 0x55907b29e950 2024-04-20T21:38:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc68c00640 10 client.26282 handle_command_reply: tid=0 2024-04-20T21:38:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 client.26282 shutdown 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 msgr2=0x55907b2a8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55907b289e20 tx=0x7efc6c02c0b0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 shutdown_connections 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55907b28aac0 0x55907b2a8810 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.119+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55907b29e950 0x55907b2a0d40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55907b2936a0 0x55907b29d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc8885e740 1 --2- 172.21.15.7:0/749330332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55907b28a1a0 0x55907b2a82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 >> 172.21.15.7:0/749330332 conn(0x55907b2848d0 msgr2=0x55907b283030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc68c00640 0 client.26282 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:38:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 shutdown_connections 2024-04-20T21:38:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.120+0000 7efc8885e740 1 -- 172.21.15.7:0/749330332 wait complete. 2024-04-20T21:38:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.121+0000 7efc8885e740 20 client.26282 trim_cache size 0 max 16384 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (218 inodes in the stack)", 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "c33ed40b-394a-4727-98ed-452aa2716278": { 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:38:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "c33ed40b-394a-4727-98ed-452aa2716278", 2024-04-20T21:38:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:38:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:38:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:38:47.124 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:38:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.122+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 msgr2=0x7efc640a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.122+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 0x7efc640a6360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efc78005fd0 tx=0x7efc78005ea0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.122+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 msgr2=0x7efc8010e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.122+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efc6c037040 tx=0x7efc6c0045e0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 shutdown_connections 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7efc640a3ea0 0x7efc640a6360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc80107260 0x7efc8010e690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc801a3810 0x7efc801a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 --2- 172.21.15.7:0/691414762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc80102190 0x7efc8010e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:38:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 >> 172.21.15.7:0/691414762 conn(0x7efc800fda10 msgr2=0x7efc800fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:38:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 shutdown_connections 2024-04-20T21:38:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:38:47.123+0000 7efc85c00640 1 -- 172.21.15.7:0/691414762 wait complete. 2024-04-20T21:38:47.157 INFO:tasks.cephfs.filesystem:scrub status for tag:c33ed40b-394a-4727-98ed-452aa2716278 - {'path': '/', 'tag': 'c33ed40b-394a-4727-98ed-452aa2716278', 'options': 'recursive,force'} 2024-04-20T21:38:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:48 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:38:48.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:48 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:48.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:48 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:48.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:48 smithi016 ceph-mon[21409]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.81k op/s 2024-04-20T21:38:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:48 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:38:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:48 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:48 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:48 smithi186 ceph-mon[22157]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.81k op/s 2024-04-20T21:38:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:48 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:38:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:48 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:48 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:48 smithi007 ceph-mon[16867]: pgmap v167: 97 pgs: 97 active+clean; 21 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.81k op/s 2024-04-20T21:38:50.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:50 smithi016 ceph-mon[21409]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-04-20T21:38:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:38:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:50 smithi186 ceph-mon[22157]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:38:50.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:50 smithi007 ceph-mon[16867]: pgmap v168: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 43 MiB/s wr, 7.60k op/s 2024-04-20T21:38:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:52 smithi016 ceph-mon[21409]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.96k op/s 2024-04-20T21:38:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:52 smithi186 ceph-mon[22157]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.96k op/s 2024-04-20T21:38:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:52 smithi007 ceph-mon[16867]: pgmap v169: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 39 MiB/s wr, 6.96k op/s 2024-04-20T21:38:54.585 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:54 smithi007 ceph-mon[16867]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 MiB/s wr, 8.48k op/s 2024-04-20T21:38:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:54 smithi016 ceph-mon[21409]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 MiB/s wr, 8.48k op/s 2024-04-20T21:38:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:54 smithi186 ceph-mon[22157]: pgmap v170: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 46 MiB/s wr, 8.48k op/s 2024-04-20T21:38:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:56 smithi016 ceph-mon[21409]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 39 MiB/s wr, 7.25k op/s 2024-04-20T21:38:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:56 smithi186 ceph-mon[22157]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 39 MiB/s wr, 7.25k op/s 2024-04-20T21:38:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:56 smithi007 ceph-mon[16867]: pgmap v171: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 39 MiB/s wr, 7.25k op/s 2024-04-20T21:38:57.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:57 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:57.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:57 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:57.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:57 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:58 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:58 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:38:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:38:58 smithi016 ceph-mon[21409]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 MiB/s wr, 6.72k op/s 2024-04-20T21:38:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:58 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:58 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:38:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:38:58 smithi186 ceph-mon[22157]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 MiB/s wr, 6.72k op/s 2024-04-20T21:38:58.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:58 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:58 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:38:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:38:58 smithi007 ceph-mon[16867]: pgmap v172: 97 pgs: 97 active+clean; 21 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 36 MiB/s wr, 6.72k op/s 2024-04-20T21:39:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:00 smithi016 ceph-mon[21409]: pgmap v173: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 7.89k op/s 2024-04-20T21:39:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:00 smithi186 ceph-mon[22157]: pgmap v173: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 7.89k op/s 2024-04-20T21:39:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:00 smithi007 ceph-mon[16867]: pgmap v173: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 7.89k op/s 2024-04-20T21:39:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:02 smithi016 ceph-mon[21409]: pgmap v174: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-20T21:39:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:02 smithi186 ceph-mon[22157]: pgmap v174: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-20T21:39:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:02 smithi007 ceph-mon[16867]: pgmap v174: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-20T21:39:04.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:04 smithi016 ceph-mon[21409]: pgmap v175: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 6.10k op/s 2024-04-20T21:39:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:04 smithi186 ceph-mon[22157]: pgmap v175: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 6.10k op/s 2024-04-20T21:39:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:04 smithi007 ceph-mon[16867]: pgmap v175: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 31 MiB/s wr, 6.10k op/s 2024-04-20T21:39:06.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:06 smithi016 ceph-mon[21409]: pgmap v176: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 4.38k op/s 2024-04-20T21:39:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:06 smithi186 ceph-mon[22157]: pgmap v176: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 4.38k op/s 2024-04-20T21:39:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:06 smithi007 ceph-mon[16867]: pgmap v176: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 4.38k op/s 2024-04-20T21:39:08.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:08 smithi016 ceph-mon[21409]: pgmap v177: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:08 smithi186 ceph-mon[22157]: pgmap v177: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:08 smithi007 ceph-mon[16867]: pgmap v177: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:10.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:10 smithi016 ceph-mon[21409]: pgmap v178: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:10 smithi186 ceph-mon[22157]: pgmap v178: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:10 smithi007 ceph-mon[16867]: pgmap v178: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 16 MiB/s wr, 3.17k op/s 2024-04-20T21:39:12.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:12 smithi016 ceph-mon[21409]: pgmap v179: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:12.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:12 smithi186 ceph-mon[22157]: pgmap v179: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:12 smithi007 ceph-mon[16867]: pgmap v179: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:14.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:14 smithi016 ceph-mon[21409]: pgmap v180: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:14.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:14 smithi186 ceph-mon[22157]: pgmap v180: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:14 smithi007 ceph-mon[16867]: pgmap v180: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 1.18k op/s 2024-04-20T21:39:16.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:16 smithi016 ceph-mon[21409]: pgmap v181: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.5 MiB/s wr, 539 op/s 2024-04-20T21:39:16.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:16 smithi186 ceph-mon[22157]: pgmap v181: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.5 MiB/s wr, 539 op/s 2024-04-20T21:39:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:16 smithi007 ceph-mon[16867]: pgmap v181: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.5 MiB/s wr, 539 op/s 2024-04-20T21:39:17.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:17.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:17.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 -- 172.21.15.7:0/1042538075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df01043a0 msgr2=0x7f3df0104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 --2- 172.21.15.7:0/1042538075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df01043a0 0x7f3df0104780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009ca0 tx=0x7f3de002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 -- 172.21.15.7:0/1042538075 shutdown_connections 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 --2- 172.21.15.7:0/1042538075 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df0109470 0x7f3df0111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 --2- 172.21.15.7:0/1042538075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df0104d50 0x7f3df0108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 --2- 172.21.15.7:0/1042538075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df01043a0 0x7f3df0104780 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.393+0000 7f3df4e00640 1 -- 172.21.15.7:0/1042538075 >> 172.21.15.7:0/1042538075 conn(0x7f3df00ffbe0 msgr2=0x7f3df0102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.394+0000 7f3df4e00640 1 -- 172.21.15.7:0/1042538075 shutdown_connections 2024-04-20T21:39:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.394+0000 7f3df4e00640 1 -- 172.21.15.7:0/1042538075 wait complete. 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.394+0000 7f3df4e00640 1 Processor -- start 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 -- start start 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df0109470 0x7f3df01a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df019a950 0x7f3df019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0114530 con 0x7f3df01043a0 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01143b0 con 0x7f3df0109470 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3df4e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01146b0 con 0x7f3df019a950 2024-04-20T21:39:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3dede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3dede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50458/0 (socket says 172.21.15.7:50458) 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3dede00640 1 -- 172.21.15.7:0/3852905106 learned_addr learned my addr 172.21.15.7:0/3852905106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.395+0000 7f3dee800640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df019a950 0x7f3df019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3ded400640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df0109470 0x7f3df01a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 -- 172.21.15.7:0/3852905106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df019a950 msgr2=0x7f3df019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df019a950 0x7f3df019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 -- 172.21.15.7:0/3852905106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df0109470 msgr2=0x7f3df01a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df0109470 0x7f3df01a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 -- 172.21.15.7:0/3852905106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df019b470 con 0x7f3df01043a0 2024-04-20T21:39:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3dede00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005e30 tx=0x7f3de0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.396+0000 7f3de6a00640 1 -- 172.21.15.7:0/3852905106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0041020 con 0x7f3df01043a0 2024-04-20T21:39:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.397+0000 7f3de6a00640 1 -- 172.21.15.7:0/3852905106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de0034cd0 con 0x7f3df01043a0 2024-04-20T21:39:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.397+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df019b700 con 0x7f3df01043a0 2024-04-20T21:39:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.397+0000 7f3de6a00640 1 -- 172.21.15.7:0/3852905106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0039040 con 0x7f3df01043a0 2024-04-20T21:39:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.397+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a78f0 con 0x7f3df01043a0 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 Processor -- start 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3de6a00640 1 -- 172.21.15.7:0/3852905106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3de0044020 con 0x7f3df01043a0 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 -- start start 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f4186a0 0x55c65f41aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c65f375150 con 0x55c65f4186a0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c65f31b370 con 0x55c65f40fac0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.400+0000 7f3df7b81740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c65f371110 con 0x55c65f40f1a0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dee800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dee800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44414/0 (socket says 172.21.15.7:44414) 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60320/0 (socket says 172.21.15.7:60320) 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dee800640 1 -- 172.21.15.7:0/35179528 learned_addr learned my addr 172.21.15.7:0/35179528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3ded400640 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f4186a0 0x55c65f41aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 -- 172.21.15.7:0/35179528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 msgr2=0x55c65f40f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 -- 172.21.15.7:0/35179528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f4186a0 msgr2=0x55c65f41aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f4186a0 0x55c65f41aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dede00640 1 -- 172.21.15.7:0/35179528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65f1517d0 con 0x55c65f40fac0 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.401+0000 7f3dee800640 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3dede00640 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3de00efcb0 tx=0x7f3de00f8510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bf9ef263bd83b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3de6a00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 0x7f3dd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3ddf400640 1 -- 172.21.15.7:0/35179528 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00bdd10 con 0x55c65f40fac0 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3ddf400640 1 -- 172.21.15.7:0/35179528 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de00ff030 con 0x55c65f40fac0 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3ddf400640 1 -- 172.21.15.7:0/35179528 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00bdef0 con 0x55c65f40fac0 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3dee800640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 0x7f3dd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.402+0000 7f3de6a00640 1 -- 172.21.15.7:0/3852905106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3de00f2400 con 0x7f3df01043a0 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- 172.21.15.7:0/35179528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 msgr2=0x55c65f418160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3de00efcb0 tx=0x7f3de00f8510 comp rx=0 tx=0).stop 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- 172.21.15.7:0/35179528 shutdown_connections 2024-04-20T21:39:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40fac0 0x55c65f418160 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f4186a0 0x55c65f41aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 --2- 172.21.15.7:0/35179528 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f40f1a0 0x55c65f40f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- 172.21.15.7:0/35179528 >> 172.21.15.7:0/35179528 conn(0x55c65f4098d0 msgr2=0x55c65f409cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3dee800640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 0x7f3dd40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3de80124a0 tx=0x7f3de8005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- 172.21.15.7:0/35179528 shutdown_connections 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- 172.21.15.7:0/35179528 wait complete. 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.403+0000 7f3df7b81740 1 -- start start 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 Processor -- start 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f415b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f416050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c65f41b250 con 0x55c65f40fac0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c65f32e740 con 0x55c65f40f1a0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55c65f41b3d0 con 0x55c65f4186a0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.404+0000 7f3df7b81740 10 client.0 _ll_register_callbacks cb 0x55c65f329b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3ded400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f415b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60324/0 (socket says 172.21.15.7:60324) 2024-04-20T21:39:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f416050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dede00640 1 -- 172.21.15.7:0/22837820 learned_addr learned my addr 172.21.15.7:0/22837820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 -- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 msgr2=0x55c65f4155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 -- 172.21.15.7:0/22837820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 msgr2=0x55c65f415b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f415b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 -- 172.21.15.7:0/22837820 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65f0873f0 con 0x55c65f4186a0 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dede00640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dee800640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f416050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3de8012d30 tx=0x7f3de8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf7f19d704f9e27a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.405+0000 7f3dd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8005020 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de00ff030 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3de80051c0 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3de80053c0 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3df7b81740 10 client.26254 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c65f0470f0 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.406+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de00ff030 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3de8027070 con 0x55c65f4186a0 2024-04-20T21:39:17.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 10 client.26254 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 20 client.26254 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f428750 0x55c65f42abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 4 client.26254 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c65f382b70 con 0x55c65f428750 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.407+0000 7f3dede00640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f428750 0x55c65f42abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.408+0000 7f3dede00640 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f428750 0x55c65f42abf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18be116369d74c4b server_cookie=8984e5e4c210ba27 in_seq=0 out_seq=0 2024-04-20T21:39:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.408+0000 7f3dd3e00640 10 client.26254 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3dd3e00640 1 -- 172.21.15.7:0/22837820 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c65f382b70 con 0x55c65f428750 2024-04-20T21:39:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3dd3e00640 10 client.26254 handle_command_reply: tid=0 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 client.26254 shutdown 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 msgr2=0x55c65f416050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f416050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3de8012d30 tx=0x7f3de8004730 comp rx=0 tx=0).stop 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 shutdown_connections 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f428750 0x55c65f42abf0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f416050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f415b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 --2- 172.21.15.7:0/22837820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f4155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3dd3e00640 0 client.26254 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 >> 172.21.15.7:0/22837820 conn(0x55c65f4098d0 msgr2=0x55c65f40b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 shutdown_connections 2024-04-20T21:39:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.409+0000 7f3df7b81740 1 -- 172.21.15.7:0/22837820 wait complete. 2024-04-20T21:39:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.410+0000 7f3df7b81740 20 client.26254 trim_cache size 0 max 16384 2024-04-20T21:39:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 Processor -- start 2024-04-20T21:39:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 -- start start 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f424110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f414480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2760 con 0x55c65f40fac0 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b0020 con 0x55c65f40f1a0 2024-04-20T21:39:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3df7b81740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01ade90 con 0x55c65f4186a0 2024-04-20T21:39:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3ded400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f424110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.429+0000 7f3dee800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f414480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dee800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f414480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44438/0 (socket says 172.21.15.7:44438) 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60332/0 (socket says 172.21.15.7:60332) 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dee800640 1 -- 172.21.15.7:0/1633929404 learned_addr learned my addr 172.21.15.7:0/1633929404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3ded400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f424110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 -- 172.21.15.7:0/1633929404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 msgr2=0x55c65f414480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f414480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 -- 172.21.15.7:0/1633929404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 msgr2=0x55c65f424110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f424110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 -- 172.21.15.7:0/1633929404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c65f41aa90 con 0x55c65f40f1a0 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3dede00640 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3de00be520 tx=0x7f3de002c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d0fa51cc805abf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3ddea00640 1 -- 172.21.15.7:0/1633929404 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00be0d0 con 0x55c65f40f1a0 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3ddea00640 1 -- 172.21.15.7:0/1633929404 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de00ff040 con 0x55c65f40f1a0 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.430+0000 7f3ddea00640 1 -- 172.21.15.7:0/1633929404 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00be360 con 0x55c65f40f1a0 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 -- 172.21.15.7:0/1633929404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 msgr2=0x55c65f417d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3de00be520 tx=0x7f3de002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 -- 172.21.15.7:0/1633929404 shutdown_connections 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f414480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f424110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 --2- 172.21.15.7:0/1633929404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f417d00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 -- 172.21.15.7:0/1633929404 >> 172.21.15.7:0/1633929404 conn(0x55c65f4098d0 msgr2=0x55c65f413d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 -- 172.21.15.7:0/1633929404 shutdown_connections 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.431+0000 7f3df7b81740 1 -- 172.21.15.7:0/1633929404 wait complete. 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 -- start start 2024-04-20T21:39:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 Processor -- start 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f420920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f41c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01ae4e0 con 0x55c65f40fac0 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01af950 con 0x55c65f40f1a0 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01aebb0 con 0x55c65f4186a0 2024-04-20T21:39:17.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3df7b81740 10 client.0 _ll_register_callbacks cb 0x7f3de8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.432+0000 7f3ded400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3dede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f420920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50500/0 (socket says 172.21.15.7:50500) 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 -- 172.21.15.7:0/900394334 learned_addr learned my addr 172.21.15.7:0/900394334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3dee800640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f41c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 -- 172.21.15.7:0/900394334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 msgr2=0x55c65f41c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f41c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 -- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 msgr2=0x55c65f420920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f420920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 -- 172.21.15.7:0/900394334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65f41c8c0 con 0x55c65f40fac0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3dede00640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f420920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3ded400640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8005fd0 tx=0x7f3dd8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0be3eae513e2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.433+0000 7f3dd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8021030 con 0x55c65f40fac0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de00ff040 con 0x55c65f40fac0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3dd8010c30 con 0x55c65f40fac0 2024-04-20T21:39:17.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3dd8010e70 con 0x55c65f40fac0 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3df7b81740 10 client.16092 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c65f48ed30 con 0x55c65f40fac0 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.434+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de00ff040 con 0x55c65f40fac0 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3dd80050b0 con 0x55c65f40fac0 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 10 client.16092 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 10 client.16092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 20 client.16092 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f48f510 0x55c65f491930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 4 client.16092 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:17.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c65f37c9e0 con 0x55c65f48f510 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3dede00640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f48f510 0x55c65f491930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3dede00640 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f48f510 0x55c65f491930 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b6e1abf05bad7a server_cookie=5cdfc3c859df9752 in_seq=0 out_seq=0 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.435+0000 7f3dd3e00640 10 client.16092 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3dd3e00640 1 -- 172.21.15.7:0/900394334 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c65f37c9e0 con 0x55c65f48f510 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3dd3e00640 10 client.16092 handle_command_reply: tid=0 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 client.16092 shutdown 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 msgr2=0x55c65f41bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8005fd0 tx=0x7f3dd8012b60 comp rx=0 tx=0).stop 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 shutdown_connections 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55c65f48f510 0x55c65f491930 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55c65f4186a0 0x55c65f41c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c65f40fac0 0x55c65f41bc40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 --2- 172.21.15.7:0/900394334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c65f40f1a0 0x55c65f420920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 >> 172.21.15.7:0/900394334 conn(0x55c65f4098d0 msgr2=0x55c65f414600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3dd3e00640 0 client.16092 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 shutdown_connections 2024-04-20T21:39:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 1 -- 172.21.15.7:0/900394334 wait complete. 2024-04-20T21:39:17.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.436+0000 7f3df7b81740 20 client.16092 trim_cache size 0 max 16384 2024-04-20T21:39:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:39:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:39:17.443 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 msgr2=0x7f3dd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 0x7f3dd40a61e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3de80124a0 tx=0x7f3de8005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 msgr2=0x7f3df01a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005e30 tx=0x7f3de0032f40 comp rx=0 tx=0).stop 2024-04-20T21:39:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 shutdown_connections 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3dd40a3d20 0x7f3dd40a61e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df0109470 0x7f3df01a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3df019a950 0x7f3df019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 --2- 172.21.15.7:0/3852905106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df01043a0 0x7f3df01a07a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 >> 172.21.15.7:0/3852905106 conn(0x7f3df00ffbe0 msgr2=0x7f3df0100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 shutdown_connections 2024-04-20T21:39:17.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.437+0000 7f3df4e00640 1 -- 172.21.15.7:0/3852905106 wait complete. 2024-04-20T21:39:17.455 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:39:17.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.668+0000 7f57a7e00640 1 -- 172.21.15.7:0/3732236671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a81002c0 msgr2=0x7f57a81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.668+0000 7f57a7e00640 1 --2- 172.21.15.7:0/3732236671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a81002c0 0x7f57a81006a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5798009ca0 tx=0x7f579802c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 -- 172.21.15.7:0/3732236671 shutdown_connections 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 --2- 172.21.15.7:0/3732236671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8100c70 0x7f57a8104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 --2- 172.21.15.7:0/3732236671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a8105390 0x7f57a810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 --2- 172.21.15.7:0/3732236671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a81002c0 0x7f57a81006a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 -- 172.21.15.7:0/3732236671 >> 172.21.15.7:0/3732236671 conn(0x7f57a8076e30 msgr2=0x7f57a80fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 -- 172.21.15.7:0/3732236671 shutdown_connections 2024-04-20T21:39:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.669+0000 7f57a7e00640 1 -- 172.21.15.7:0/3732236671 wait complete. 2024-04-20T21:39:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.670+0000 7f57a7e00640 1 Processor -- start 2024-04-20T21:39:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.670+0000 7f57a7e00640 1 -- start start 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.670+0000 7f57a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.670+0000 7f57a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8105390 0x7f57a81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.670+0000 7f57a7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 0x7f57a819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8112200 con 0x7f57a81002c0 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8112080 con 0x7f57a8105390 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8112380 con 0x7f57a819a760 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50520/0 (socket says 172.21.15.7:50520) 2024-04-20T21:39:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 -- 172.21.15.7:0/545956162 learned_addr learned my addr 172.21.15.7:0/545956162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6000640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8105390 0x7f57a81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a7400640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 0x7f57a819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 -- 172.21.15.7:0/545956162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 msgr2=0x7f57a819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 0x7f57a819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 -- 172.21.15.7:0/545956162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8105390 msgr2=0x7f57a81a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8105390 0x7f57a81a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 -- 172.21.15.7:0/545956162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a819b360 con 0x7f57a81002c0 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a7400640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 0x7f57a819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.671+0000 7f57a6a00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5798005e30 tx=0x7f5798032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.672+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798041020 con 0x7f57a81002c0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.672+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798034cd0 con 0x7f57a81002c0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.672+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a819b5f0 con 0x7f57a81002c0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.672+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798039040 con 0x7f57a81002c0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.672+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a81a7730 con 0x7f57a81002c0 2024-04-20T21:39:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.673+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a8101790 con 0x7f57a81002c0 2024-04-20T21:39:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.675+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5798044020 con 0x7f57a81002c0 2024-04-20T21:39:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.678+0000 7f579f400640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 0x7f578c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.678+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f57980f2430 con 0x7f57a81002c0 2024-04-20T21:39:17.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.678+0000 7f57a6000640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 0x7f578c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:17.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.679+0000 7f57a6000640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 0x7f578c0a61d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57940095f0 tx=0x7f5794005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:17.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.683+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f579804b020 con 0x7f57a81002c0 2024-04-20T21:39:17.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.933+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57a8100c70 con 0x7f57a81002c0 2024-04-20T21:39:17.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.935+0000 7f579f400640 1 -- 172.21.15.7:0/545956162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f57980bebd0 con 0x7f57a81002c0 2024-04-20T21:39:17.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:39:17.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:39:17.934 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:39:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 msgr2=0x7f578c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 0x7f578c0a61d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f57940095f0 tx=0x7f5794005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 msgr2=0x7f57a81a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:17.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5798005e30 tx=0x7f5798032c90 comp rx=0 tx=0).stop 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 shutdown_connections 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f578c0a3d10 0x7f578c0a61d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a8105390 0x7f57a81a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f57a819a760 0x7f57a819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 --2- 172.21.15.7:0/545956162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a81002c0 0x7f57a81a06a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 >> 172.21.15.7:0/545956162 conn(0x7f57a8076e30 msgr2=0x7f57a81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 shutdown_connections 2024-04-20T21:39:17.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:17.938+0000 7f57a7e00640 1 -- 172.21.15.7:0/545956162 wait complete. 2024-04-20T21:39:17.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:39:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 -- 172.21.15.7:0/103563412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88104b60 msgr2=0x7ffa88108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 --2- 172.21.15.7:0/103563412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88104b60 0x7ffa88108ba0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009860 tx=0x7ffa7402bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 -- 172.21.15.7:0/103563412 shutdown_connections 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 --2- 172.21.15.7:0/103563412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa88109280 0x7ffa88110e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 --2- 172.21.15.7:0/103563412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88104b60 0x7ffa88108ba0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 --2- 172.21.15.7:0/103563412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 0x7ffa88104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.181+0000 7ffa8f000640 1 -- 172.21.15.7:0/103563412 >> 172.21.15.7:0/103563412 conn(0x7ffa880ffa30 msgr2=0x7ffa88101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.182+0000 7ffa8f000640 1 -- 172.21.15.7:0/103563412 shutdown_connections 2024-04-20T21:39:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.182+0000 7ffa8f000640 1 -- 172.21.15.7:0/103563412 wait complete. 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.182+0000 7ffa8f000640 1 Processor -- start 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 -- start start 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 0x7ffa881a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 0x7ffa881a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88114310 con 0x7ffa88109280 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88114190 con 0x7ffa881041b0 2024-04-20T21:39:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8f000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa88114490 con 0x7ffa8819a6d0 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 0x7ffa881a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44498/0 (socket says 172.21.15.7:44498) 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.183+0000 7ffa8e600640 1 -- 172.21.15.7:0/3299118249 learned_addr learned my addr 172.21.15.7:0/3299118249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 0x7ffa881a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 -- 172.21.15.7:0/3299118249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 msgr2=0x7ffa881a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 0x7ffa881a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 -- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 msgr2=0x7ffa881a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 0x7ffa881a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 -- 172.21.15.7:0/3299118249 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8819b2d0 con 0x7ffa8819a6d0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8d200640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 0x7ffa881a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa8e600640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000eb10 tx=0x7ffa8000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.184+0000 7ffa86000640 1 -- 172.21.15.7:0/3299118249 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80004b20 con 0x7ffa8819a6d0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.185+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa881a7360 con 0x7ffa8819a6d0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.185+0000 7ffa86000640 1 -- 172.21.15.7:0/3299118249 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa80004cc0 con 0x7ffa8819a6d0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.185+0000 7ffa86000640 1 -- 172.21.15.7:0/3299118249 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80010760 con 0x7ffa8819a6d0 2024-04-20T21:39:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.185+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa881a7760 con 0x7ffa8819a6d0 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.188+0000 7ffa933bc740 1 Processor -- start 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.188+0000 7ffa86000640 1 -- 172.21.15.7:0/3299118249 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ffa80005020 con 0x7ffa8819a6d0 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 -- start start 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199a90 0x55ed4d1a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d0fe050 con 0x55ed4d199a90 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d0a4490 con 0x55ed4d1a2670 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa933bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d0fa010 con 0x55ed4d199170 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa8d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa8d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44506/0 (socket says 172.21.15.7:44506) 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa8d200640 1 -- 172.21.15.7:0/2925769174 learned_addr learned my addr 172.21.15.7:0/2925769174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199a90 0x55ed4d1a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.189+0000 7ffa8d200640 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 -- 172.21.15.7:0/2925769174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d1a2670 msgr2=0x55ed4d1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 -- 172.21.15.7:0/2925769174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199a90 msgr2=0x55ed4d1a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199a90 0x55ed4d1a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 -- 172.21.15.7:0/2925769174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4d0ac070 con 0x55ed4d199170 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8d200640 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005e00 tx=0x7ffa7402bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50265c394c496def server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa86000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 0x7ffa700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa7aa00640 1 -- 172.21.15.7:0/2925769174 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74039040 con 0x55ed4d199170 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa7aa00640 1 -- 172.21.15.7:0/2925769174 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7402c510 con 0x55ed4d199170 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa7aa00640 1 -- 172.21.15.7:0/2925769174 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740328a0 con 0x55ed4d199170 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa86000640 1 -- 172.21.15.7:0/3299118249 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ffa800d0e40 con 0x7ffa8819a6d0 2024-04-20T21:39:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.190+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 0x7ffa700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 -- 172.21.15.7:0/2925769174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 msgr2=0x55ed4d199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffa74005e00 tx=0x7ffa7402bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 -- 172.21.15.7:0/2925769174 shutdown_connections 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199a90 0x55ed4d1a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 --2- 172.21.15.7:0/2925769174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199170 0x55ed4d199550 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 -- 172.21.15.7:0/2925769174 >> 172.21.15.7:0/2925769174 conn(0x55ed4d1938a0 msgr2=0x55ed4d193c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 -- 172.21.15.7:0/2925769174 shutdown_connections 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 0x7ffa700a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55ed4d198c40 tx=0x7ffa7c009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.191+0000 7ffa933bc740 1 -- 172.21.15.7:0/2925769174 wait complete. 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 -- start start 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 Processor -- start 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d1a5400 con 0x55ed4d1a2670 2024-04-20T21:39:18.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d0b7860 con 0x55ed4d199170 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed4d1a5580 con 0x55ed4d199a90 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.192+0000 7ffa933bc740 10 client.0 _ll_register_callbacks cb 0x55ed4d085fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50544/0 (socket says 172.21.15.7:50544) 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8e600640 1 -- 172.21.15.7:0/3983441076 learned_addr learned my addr 172.21.15.7:0/3983441076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8d200640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 -- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 msgr2=0x55ed4d19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 -- 172.21.15.7:0/3983441076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 msgr2=0x55ed4d1a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8dc00640 1 -- 172.21.15.7:0/3983441076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4d0bb950 con 0x55ed4d199170 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.193+0000 7ffa8e600640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19f820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005fd0 tx=0x7ffa7c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93eb06c186675bb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa6f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c02e060 con 0x55ed4d199170 2024-04-20T21:39:18.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ffa7c018030 con 0x55ed4d199170 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7402c510 con 0x55ed4d199170 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c009e00 con 0x55ed4d199170 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa933bc740 10 client.26318 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.194+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed4d089820 con 0x55ed4d199170 2024-04-20T21:39:18.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7402c510 con 0x55ed4d199170 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ffa7c032670 con 0x55ed4d199170 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 10 client.26318 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 10 client.26318 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 20 client.26318 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d1b27e0 0x55ed4d1b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 4 client.26318 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.195+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed4d10ba70 con 0x55ed4d1b27e0 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.196+0000 7ffa8d200640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d1b27e0 0x55ed4d1b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.196+0000 7ffa8d200640 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d1b27e0 0x55ed4d1b4bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93f04b5f3a0a0b66 server_cookie=3e2d01aa80dd7ddc in_seq=0 out_seq=0 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.196+0000 7ffa6f400640 10 client.26318 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa6f400640 1 -- 172.21.15.7:0/3983441076 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed4d10ba70 con 0x55ed4d1b27e0 2024-04-20T21:39:18.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa6f400640 10 client.26318 handle_command_reply: tid=0 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 client.26318 shutdown 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 msgr2=0x55ed4d19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19f820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c005fd0 tx=0x7ffa7c004730 comp rx=0 tx=0).stop 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 shutdown_connections 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d1a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d1b27e0 0x55ed4d1b4bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 --2- 172.21.15.7:0/3983441076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19f820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa6f400640 0 client.26318 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:18.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.197+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 >> 172.21.15.7:0/3983441076 conn(0x55ed4d1938a0 msgr2=0x55ed4d1955f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.198+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 shutdown_connections 2024-04-20T21:39:18.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.198+0000 7ffa933bc740 1 -- 172.21.15.7:0/3983441076 wait complete. 2024-04-20T21:39:18.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.198+0000 7ffa933bc740 20 client.26318 trim_cache size 0 max 16384 2024-04-20T21:39:18.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 Processor -- start 2024-04-20T21:39:18.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 -- start start 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 0x55ed4d1ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d1a2670 0x55ed4d1ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881b2490 con 0x55ed4d199170 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881afd50 con 0x55ed4d199a90 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa933bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881adbc0 con 0x55ed4d1a2670 2024-04-20T21:39:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50548/0 (socket says 172.21.15.7:50548) 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.213+0000 7ffa8dc00640 1 -- 172.21.15.7:0/1057218682 learned_addr learned my addr 172.21.15.7:0/1057218682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8d200640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 0x55ed4d1ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8e600640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d1a2670 0x55ed4d1ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 -- 172.21.15.7:0/1057218682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d1a2670 msgr2=0x55ed4d1ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d1a2670 0x55ed4d1ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 -- 172.21.15.7:0/1057218682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 msgr2=0x55ed4d1ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 0x55ed4d1ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 -- 172.21.15.7:0/1057218682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed4ce103f0 con 0x55ed4d199170 2024-04-20T21:39:18.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8d200640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 0x55ed4d1ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:18.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.214+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009530 tx=0x7ffa7c01ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48626138c43154ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa7a000640 1 -- 172.21.15.7:0/1057218682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c032410 con 0x55ed4d199170 2024-04-20T21:39:18.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa7a000640 1 -- 172.21.15.7:0/1057218682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c0325b0 con 0x55ed4d199170 2024-04-20T21:39:18.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa7a000640 1 -- 172.21.15.7:0/1057218682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c029640 con 0x55ed4d199170 2024-04-20T21:39:18.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 -- 172.21.15.7:0/1057218682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 msgr2=0x55ed4d1a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009530 tx=0x7ffa7c01ee20 comp rx=0 tx=0).stop 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 -- 172.21.15.7:0/1057218682 shutdown_connections 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199a90 0x55ed4d1ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d1a2670 0x55ed4d1ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 --2- 172.21.15.7:0/1057218682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d199170 0x55ed4d1a1f50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 -- 172.21.15.7:0/1057218682 >> 172.21.15.7:0/1057218682 conn(0x55ed4d1938a0 msgr2=0x55ed4d1a0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 -- 172.21.15.7:0/1057218682 shutdown_connections 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.215+0000 7ffa933bc740 1 -- 172.21.15.7:0/1057218682 wait complete. 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.216+0000 7ffa933bc740 1 -- start start 2024-04-20T21:39:18.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.216+0000 7ffa933bc740 1 Processor -- start 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.216+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.216+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.216+0000 7ffa933bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa933bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881ae210 con 0x55ed4d1a2670 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa933bc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881af680 con 0x55ed4d199170 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa933bc740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffa881ae8e0 con 0x55ed4d199a90 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa933bc740 10 client.0 _ll_register_callbacks cb 0x7ffa7c01e7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50556/0 (socket says 172.21.15.7:50556) 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 -- 172.21.15.7:0/712981184 learned_addr learned my addr 172.21.15.7:0/712981184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8d200640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 -- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 msgr2=0x55ed4d19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 -- 172.21.15.7:0/712981184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 msgr2=0x55ed4d19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8e600640 1 -- 172.21.15.7:0/712981184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed4ceda7d0 con 0x55ed4d1a2670 2024-04-20T21:39:18.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.217+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa8e600640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffa800997a0 tx=0x7ffa80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8616e1127809679 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa6f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80005510 con 0x55ed4d1a2670 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ffa8009c220 con 0x55ed4d1a2670 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c0325b0 con 0x55ed4d1a2670 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa8009cdd0 con 0x55ed4d1a2670 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa933bc740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed4d219040 con 0x55ed4d1a2670 2024-04-20T21:39:18.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.218+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa7c0325b0 con 0x55ed4d1a2670 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ffa800bd140 con 0x55ed4d1a2670 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 10 client.16116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 20 client.16116 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d219760 0x55ed4d21bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 4 client.16116 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed4d1058e0 con 0x55ed4d219760 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.219+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d219760 0x55ed4d21bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa8dc00640 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d219760 0x55ed4d21bb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90e02c499778c9e5 server_cookie=a1d91de9e313d533 in_seq=0 out_seq=0 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa6f400640 10 client.16116 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa6f400640 1 -- 172.21.15.7:0/712981184 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed4d1058e0 con 0x55ed4d219760 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa6f400640 10 client.16116 handle_command_reply: tid=0 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 client.16116 shutdown 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 msgr2=0x55ed4d19d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffa800997a0 tx=0x7ffa80004a40 comp rx=0 tx=0).stop 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 shutdown_connections 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed4d1a2670 0x55ed4d19d150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed4d199a90 0x55ed4d19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55ed4d219760 0x55ed4d21bb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 --2- 172.21.15.7:0/712981184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed4d199170 0x55ed4d19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 >> 172.21.15.7:0/712981184 conn(0x55ed4d1938a0 msgr2=0x55ed4d1a0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa6f400640 0 client.16116 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 shutdown_connections 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.220+0000 7ffa933bc740 1 -- 172.21.15.7:0/712981184 wait complete. 2024-04-20T21:39:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa933bc740 20 client.16116 trim_cache size 0 max 16384 2024-04-20T21:39:18.227 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 msgr2=0x7ffa700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 0x7ffa700a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55ed4d198c40 tx=0x7ffa7c009260 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 msgr2=0x7ffa8819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000eb10 tx=0x7ffa8000c2d0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 shutdown_connections 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa8819a6d0 0x7ffa8819ab50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa88109280 0x7ffa881a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffa700a3d20 0x7ffa700a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 --2- 172.21.15.7:0/3299118249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa881041b0 0x7ffa881a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 >> 172.21.15.7:0/3299118249 conn(0x7ffa880ffa30 msgr2=0x7ffa88100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.221+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 shutdown_connections 2024-04-20T21:39:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.222+0000 7ffa8f000640 1 -- 172.21.15.7:0/3299118249 wait complete. 2024-04-20T21:39:18.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:39:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 -- 172.21.15.7:0/2264330984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 msgr2=0x7f29401023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 --2- 172.21.15.7:0/2264330984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401023b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2930009860 tx=0x7f293002bb70 comp rx=0 tx=0).stop 2024-04-20T21:39:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 -- 172.21.15.7:0/2264330984 shutdown_connections 2024-04-20T21:39:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 --2- 172.21.15.7:0/2264330984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f294010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 --2- 172.21.15.7:0/2264330984 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2940102980 0x7f29401069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 --2- 172.21.15.7:0/2264330984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401023b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 -- 172.21.15.7:0/2264330984 >> 172.21.15.7:0/2264330984 conn(0x7f29400fd8b0 msgr2=0x7f29400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 -- 172.21.15.7:0/2264330984 shutdown_connections 2024-04-20T21:39:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.456+0000 7f293f200640 1 -- 172.21.15.7:0/2264330984 wait complete. 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.457+0000 7f293f200640 1 Processor -- start 2024-04-20T21:39:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 -- start start 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f294019a770 0x7f294019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2940112150 con 0x7f29401070a0 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2940111fd0 con 0x7f2940101fd0 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29401122d0 con 0x7f294019a770 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50584/0 (socket says 172.21.15.7:50584) 2024-04-20T21:39:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.458+0000 7f293d400640 1 -- 172.21.15.7:0/928312892 learned_addr learned my addr 172.21.15.7:0/928312892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293de00640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293e800640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f294019a770 0x7f294019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 -- 172.21.15.7:0/928312892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f294019a770 msgr2=0x7f294019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f294019a770 0x7f294019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 -- 172.21.15.7:0/928312892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 msgr2=0x7f29401a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 -- 172.21.15.7:0/928312892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019b350 con 0x7f29401070a0 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f293d400640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f292800e9c0 tx=0x7f292800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.459+0000 7f2936a00640 1 -- 172.21.15.7:0/928312892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292800ccc0 con 0x7f29401070a0 2024-04-20T21:39:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.460+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29401a7430 con 0x7f29401070a0 2024-04-20T21:39:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.460+0000 7f2936a00640 1 -- 172.21.15.7:0/928312892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2928005500 con 0x7f29401070a0 2024-04-20T21:39:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.460+0000 7f2936a00640 1 -- 172.21.15.7:0/928312892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928010690 con 0x7f29401070a0 2024-04-20T21:39:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.460+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a77e0 con 0x7f29401070a0 2024-04-20T21:39:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.463+0000 7f2947c36740 1 Processor -- start 2024-04-20T21:39:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.463+0000 7f2947c36740 1 -- start start 2024-04-20T21:39:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.463+0000 7f2936a00640 1 -- 172.21.15.7:0/928312892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f29280046c0 con 0x7f29401070a0 2024-04-20T21:39:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f7e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563ab2eda050 con 0x563ab2f75170 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ab2e80490 con 0x563ab2f7e670 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f2947c36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563ab2ed6010 con 0x563ab2f75a90 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f293e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f293de00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f7e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f293d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f293e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50600/0 (socket says 172.21.15.7:50600) 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.464+0000 7f293e800640 1 -- 172.21.15.7:0/2372692705 learned_addr learned my addr 172.21.15.7:0/2372692705 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 -- 172.21.15.7:0/2372692705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 msgr2=0x563ab2f7e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f7e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 -- 172.21.15.7:0/2372692705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 msgr2=0x563ab2f80a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f80a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 -- 172.21.15.7:0/2372692705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab2e88070 con 0x563ab2f75170 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f293e800640 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2938009f50 tx=0x7f29380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58aec1e70fe9350 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f292f400640 1 -- 172.21.15.7:0/2372692705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938004d20 con 0x563ab2f75170 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f292f400640 1 -- 172.21.15.7:0/2372692705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2938004050 con 0x563ab2f75170 2024-04-20T21:39:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f292f400640 1 -- 172.21.15.7:0/2372692705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938005280 con 0x563ab2f75170 2024-04-20T21:39:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.465+0000 7f2936a00640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 0x7f29240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2936a00640 1 -- 172.21.15.7:0/928312892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f29280d09e0 con 0x7f29401070a0 2024-04-20T21:39:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f293de00640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 0x7f29240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 -- 172.21.15.7:0/2372692705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 msgr2=0x563ab2f75550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2938009f50 tx=0x7f29380118d0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 -- 172.21.15.7:0/2372692705 shutdown_connections 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f80a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f7e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 --2- 172.21.15.7:0/2372692705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f75550 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 -- 172.21.15.7:0/2372692705 >> 172.21.15.7:0/2372692705 conn(0x563ab2f6f8a0 msgr2=0x563ab2f6fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.466+0000 7f2947c36740 1 -- 172.21.15.7:0/2372692705 shutdown_connections 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.467+0000 7f2947c36740 1 -- 172.21.15.7:0/2372692705 wait complete. 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.467+0000 7f293de00640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 0x7f29240a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f293002c180 tx=0x7f29300058d0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.467+0000 7f2947c36740 1 -- start start 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.467+0000 7f2947c36740 1 Processor -- start 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f7b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563ab2f81460 con 0x563ab2f75a90 2024-04-20T21:39:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ab2e93860 con 0x563ab2f75170 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563ab2f815e0 con 0x563ab2f7e670 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f2947c36740 10 client.0 _ll_register_callbacks cb 0x563ab2e61fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f293d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f293de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f7b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f293d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50610/0 (socket says 172.21.15.7:50610) 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f293d400640 1 -- 172.21.15.7:0/639301575 learned_addr learned my addr 172.21.15.7:0/639301575 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.468+0000 7f293e800640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 -- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 msgr2=0x563ab2f7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 -- 172.21.15.7:0/639301575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 msgr2=0x563ab2f7b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f7b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 -- 172.21.15.7:0/639301575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab2e97950 con 0x563ab2f75a90 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293de00640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f7b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f293d400640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f292809c6d0 tx=0x7f29280d4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e805fb7a24a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2923e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292809c7a0 con 0x563ab2f75a90 2024-04-20T21:39:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2938004050 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f29280a53b0 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f292809c330 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2947c36740 10 client.16128 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.469+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ab2e65820 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2938004050 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f29280bbb90 con 0x563ab2f75a90 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 10 client.16128 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 10 client.16128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 20 client.16128 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2f8e680 0x563ab2f90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 4 client.16128 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.470+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ab2ee7a70 con 0x563ab2f8e680 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.471+0000 7f293de00640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2f8e680 0x563ab2f90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.471+0000 7f293de00640 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2f8e680 0x563ab2f90a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ce35dd2f2bde057 server_cookie=1eaf6fca24eb512a in_seq=0 out_seq=0 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.471+0000 7f2923e00640 10 client.16128 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2923e00640 1 -- 172.21.15.7:0/639301575 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ab2ee7a70 con 0x563ab2f8e680 2024-04-20T21:39:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2923e00640 10 client.16128 handle_command_reply: tid=0 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 client.16128 shutdown 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 msgr2=0x563ab2f7bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f292809c6d0 tx=0x7f29280d4120 comp rx=0 tx=0).stop 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 shutdown_connections 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2f8e680 0x563ab2f90a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7bbf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2923e00640 0 client.16128 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 --2- 172.21.15.7:0/639301575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f7b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.472+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 >> 172.21.15.7:0/639301575 conn(0x563ab2f6f8a0 msgr2=0x563ab2f715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.473+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 shutdown_connections 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.473+0000 7f2947c36740 1 -- 172.21.15.7:0/639301575 wait complete. 2024-04-20T21:39:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.473+0000 7f2947c36740 20 client.16128 trim_cache size 0 max 16384 2024-04-20T21:39:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.487+0000 7f2947c36740 1 Processor -- start 2024-04-20T21:39:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 -- start start 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29401b2440 con 0x563ab2f75a90 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29401afd00 con 0x563ab2f75170 2024-04-20T21:39:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f2947c36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29401adb50 con 0x563ab2f7e670 2024-04-20T21:39:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f293d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f293de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f293de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60478/0 (socket says 172.21.15.7:60478) 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f293de00640 1 -- 172.21.15.7:0/1674265429 learned_addr learned my addr 172.21.15.7:0/1674265429 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.488+0000 7f293e800640 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 -- 172.21.15.7:0/1674265429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 msgr2=0x563ab2f7bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 -- 172.21.15.7:0/1674265429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 msgr2=0x563ab2f7dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 -- 172.21.15.7:0/1674265429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ab2bec3f0 con 0x563ab2f75170 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f293de00640 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29300041d0 tx=0x7f293003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e28a6cd45431d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.489+0000 7f292ea00640 1 -- 172.21.15.7:0/1674265429 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930039040 con 0x563ab2f75170 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f292ea00640 1 -- 172.21.15.7:0/1674265429 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930004300 con 0x563ab2f75170 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f292ea00640 1 -- 172.21.15.7:0/1674265429 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930032b70 con 0x563ab2f75170 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 -- 172.21.15.7:0/1674265429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 msgr2=0x563ab2f8a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29300041d0 tx=0x7f293003f000 comp rx=0 tx=0).stop 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 -- 172.21.15.7:0/1674265429 shutdown_connections 2024-04-20T21:39:18.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f7e670 0x563ab2f7bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75a90 0x563ab2f7dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 --2- 172.21.15.7:0/1674265429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f75170 0x563ab2f8a320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 -- 172.21.15.7:0/1674265429 >> 172.21.15.7:0/1674265429 conn(0x563ab2f6f8a0 msgr2=0x563ab2f7b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 -- 172.21.15.7:0/1674265429 shutdown_connections 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.490+0000 7f2947c36740 1 -- 172.21.15.7:0/1674265429 wait complete. 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 -- start start 2024-04-20T21:39:18.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 Processor -- start 2024-04-20T21:39:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f81df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29401ae1c0 con 0x563ab2f75170 2024-04-20T21:39:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.491+0000 7f2947c36740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29401af630 con 0x563ab2f7e670 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f2947c36740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29401ae890 con 0x563ab2f75a90 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f2947c36740 10 client.0 _ll_register_callbacks cb 0x7f29280d4220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 -- 172.21.15.7:0/668987958 learned_addr learned my addr 172.21.15.7:0/668987958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293e800640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293d400640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 -- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 msgr2=0x563ab2f81df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f81df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 -- 172.21.15.7:0/668987958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 msgr2=0x563ab2f82330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f82330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.492+0000 7f293de00640 1 -- 172.21.15.7:0/668987958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ab2cb67d0 con 0x563ab2f75170 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f293de00640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f293002c0b0 tx=0x7f2930000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a566597b43ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2923e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930060030 con 0x563ab2f75170 2024-04-20T21:39:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f29300327f0 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2930004300 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2930069030 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2947c36740 10 client.16134 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ab2ff4750 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.493+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930004300 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f29300035f0 con 0x563ab2f75170 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 10 client.16134 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 10 client.16134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 20 client.16134 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2ff4e40 0x563ab2ff7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 4 client.16134 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ab2ee18e0 con 0x563ab2ff4e40 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.494+0000 7f293d400640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2ff4e40 0x563ab2ff7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.495+0000 7f293d400640 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2ff4e40 0x563ab2ff7260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=756ae27922cb22a9 server_cookie=31ec32ca4c47537 in_seq=0 out_seq=0 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.495+0000 7f2923e00640 10 client.16134 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.495+0000 7f2923e00640 1 -- 172.21.15.7:0/668987958 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ab2ee18e0 con 0x563ab2ff4e40 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.495+0000 7f2923e00640 10 client.16134 handle_command_reply: tid=0 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.495+0000 7f2947c36740 1 client.16134 shutdown 2024-04-20T21:39:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 msgr2=0x563ab2f86990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f293002c0b0 tx=0x7f2930000bc0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 shutdown_connections 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x563ab2ff4e40 0x563ab2ff7260 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ab2f7e670 0x563ab2f82330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563ab2f75a90 0x563ab2f81df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 --2- 172.21.15.7:0/668987958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563ab2f75170 0x563ab2f86990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 >> 172.21.15.7:0/668987958 conn(0x563ab2f6f8a0 msgr2=0x563ab2f6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2923e00640 0 client.16134 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 shutdown_connections 2024-04-20T21:39:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 1 -- 172.21.15.7:0/668987958 wait complete. 2024-04-20T21:39:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.496+0000 7f2947c36740 20 client.16134 trim_cache size 0 max 16384 2024-04-20T21:39:18.502 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 msgr2=0x7f29240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 0x7f29240a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f293002c180 tx=0x7f29300058d0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 msgr2=0x7f29401a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f292800e9c0 tx=0x7f292800ee80 comp rx=0 tx=0).stop 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 shutdown_connections 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f294019a770 0x7f294019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29401070a0 0x7f29401a0b60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29240a3d20 0x7f29240a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 --2- 172.21.15.7:0/928312892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940101fd0 0x7f29401a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 >> 172.21.15.7:0/928312892 conn(0x7f29400fd8b0 msgr2=0x7f29400ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 shutdown_connections 2024-04-20T21:39:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.497+0000 7f293f200640 1 -- 172.21.15.7:0/928312892 wait complete. 2024-04-20T21:39:18.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:18 smithi007 ceph-mon[16867]: pgmap v182: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-04-20T21:39:18.511 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:18 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/545956162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:18.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:39:18.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:18 smithi016 ceph-mon[21409]: pgmap v182: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-04-20T21:39:18.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:18 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/545956162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:18.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:18 smithi186 ceph-mon[22157]: pgmap v182: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.9 KiB/s wr, 5 op/s 2024-04-20T21:39:18.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:18 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/545956162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 -- 172.21.15.7:0/3458279636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076db0 msgr2=0x7f94441126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 --2- 172.21.15.7:0/3458279636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076db0 0x7f94441126a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9430009930 tx=0x7f943002bc30 comp rx=0 tx=0).stop 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 -- 172.21.15.7:0/3458279636 shutdown_connections 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 --2- 172.21.15.7:0/3458279636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444112be0 0x7f9444114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 --2- 172.21.15.7:0/3458279636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076db0 0x7f94441126a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 --2- 172.21.15.7:0/3458279636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444076490 0x7f9444076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.732+0000 7f9443200640 1 -- 172.21.15.7:0/3458279636 >> 172.21.15.7:0/3458279636 conn(0x7f94440ffa30 msgr2=0x7f9444101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.733+0000 7f9443200640 1 -- 172.21.15.7:0/3458279636 shutdown_connections 2024-04-20T21:39:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.733+0000 7f9443200640 1 -- 172.21.15.7:0/3458279636 wait complete. 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.734+0000 7f9443200640 1 Processor -- start 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.734+0000 7f9443200640 1 -- start start 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.734+0000 7f9443200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 0x7f94441a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.734+0000 7f9443200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 0x7f94441a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.734+0000 7f9443200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444112be0 0x7f944419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9443200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9444118740 con 0x7f9444112be0 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9443200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94441185c0 con 0x7f9444076db0 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9443200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94441188c0 con 0x7f9444076490 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 0x7f94441a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 0x7f94441a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60506/0 (socket says 172.21.15.7:60506) 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 0x7f94441a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441400640 1 -- 172.21.15.7:0/619857608 learned_addr learned my addr 172.21.15.7:0/619857608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9442800640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444112be0 0x7f944419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 -- 172.21.15.7:0/619857608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 msgr2=0x7f94441a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 0x7f94441a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 -- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444112be0 msgr2=0x7f944419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444112be0 0x7f944419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.735+0000 7f9441e00640 1 -- 172.21.15.7:0/619857608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944419f420 con 0x7f9444076490 2024-04-20T21:39:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f9441e00640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 0x7f94441a4a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9434005fd0 tx=0x7f943400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f943aa00640 1 -- 172.21.15.7:0/619857608 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434009c30 con 0x7f9444076490 2024-04-20T21:39:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f943aa00640 1 -- 172.21.15.7:0/619857608 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9434009dd0 con 0x7f9444076490 2024-04-20T21:39:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944419f710 con 0x7f9444076490 2024-04-20T21:39:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f943aa00640 1 -- 172.21.15.7:0/619857608 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434010460 con 0x7f9444076490 2024-04-20T21:39:18.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.736+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441ab9d0 con 0x7f9444076490 2024-04-20T21:39:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.739+0000 7f944b98c740 1 Processor -- start 2024-04-20T21:39:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.739+0000 7f944b98c740 1 -- start start 2024-04-20T21:39:18.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.739+0000 7f943aa00640 1 -- 172.21.15.7:0/619857608 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9434003f50 con 0x7f9444076490 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b0582f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05838130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b05794050 con 0x564b05838670 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b0573a490 con 0x564b0582f170 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f944b98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564b05790010 con 0x564b0582fa90 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f9441e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05838130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f9441400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f9441400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50636/0 (socket says 172.21.15.7:50636) 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f9441400640 1 -- 172.21.15.7:0/1915956407 learned_addr learned my addr 172.21.15.7:0/1915956407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.740+0000 7f9442800640 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b0582f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 -- 172.21.15.7:0/1915956407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 msgr2=0x564b05838130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05838130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 -- 172.21.15.7:0/1915956407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 msgr2=0x564b0582f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b0582f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 -- 172.21.15.7:0/1915956407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b05742070 con 0x564b05838670 2024-04-20T21:39:18.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f9441400640 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f943002cfd0 tx=0x7f94300095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b50c348b96153f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f942f000640 1 -- 172.21.15.7:0/1915956407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430042070 con 0x564b05838670 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f943aa00640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 0x7f94280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f942f000640 1 -- 172.21.15.7:0/1915956407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943002ca80 con 0x564b05838670 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.741+0000 7f942f000640 1 -- 172.21.15.7:0/1915956407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430004f80 con 0x564b05838670 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f943aa00640 1 -- 172.21.15.7:0/619857608 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f94340d0600 con 0x7f9444076490 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f9442800640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 0x7f94280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 -- 172.21.15.7:0/1915956407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 msgr2=0x564b0583aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f943002cfd0 tx=0x7f94300095e0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 -- 172.21.15.7:0/1915956407 shutdown_connections 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b0583aa60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05838130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 --2- 172.21.15.7:0/1915956407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b0582f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 -- 172.21.15.7:0/1915956407 >> 172.21.15.7:0/1915956407 conn(0x564b058298a0 msgr2=0x564b05829c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 -- 172.21.15.7:0/1915956407 shutdown_connections 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f9442800640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 0x7f94280a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f944406b2c0 tx=0x7f943c009600 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.742+0000 7f944b98c740 1 -- 172.21.15.7:0/1915956407 wait complete. 2024-04-20T21:39:18.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.743+0000 7f944b98c740 1 -- start start 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.743+0000 7f944b98c740 1 Processor -- start 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582fa90 0x564b05835be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b05838670 0x564b05836120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b0583b400 con 0x564b0582f170 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b0574d860 con 0x564b0582fa90 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564b0583b580 con 0x564b05838670 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f944b98c740 10 client.0 _ll_register_callbacks cb 0x564b0571bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9442800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b05838670 0x564b05836120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50638/0 (socket says 172.21.15.7:50638) 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 -- 172.21.15.7:0/1480134397 learned_addr learned my addr 172.21.15.7:0/1480134397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441400640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582fa90 0x564b05835be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 -- 172.21.15.7:0/1480134397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b05838670 msgr2=0x564b05836120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b05838670 0x564b05836120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 -- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582fa90 msgr2=0x564b05835be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582fa90 0x564b05835be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.744+0000 7f9441e00640 1 -- 172.21.15.7:0/1480134397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b05751950 con 0x564b0582f170 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f9441e00640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9434009610 tx=0x7f94340982e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951aa6eabd20e56b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f9423400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434020030 con 0x564b0582f170 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f94340cdfc0 con 0x564b0582f170 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943002ca80 con 0x564b0582f170 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f943409bfa0 con 0x564b0582f170 2024-04-20T21:39:18.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f944b98c740 10 client.16146 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.745+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b0571f820 con 0x564b0582f170 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943002ca80 con 0x564b0582f170 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f943409c8a0 con 0x564b0582f170 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 10 client.16146 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 20 client.16146 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b05848580 0x564b0584a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 4 client.16146 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b057a1a70 con 0x564b05848580 2024-04-20T21:39:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.746+0000 7f9441400640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b05848580 0x564b0584a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.747+0000 7f9441400640 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b05848580 0x564b0584a970 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef22d41ff2da250 server_cookie=3d279cd172b34b16 in_seq=0 out_seq=0 2024-04-20T21:39:18.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.747+0000 7f9423400640 10 client.16146 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f9423400640 1 -- 172.21.15.7:0/1480134397 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b057a1a70 con 0x564b05848580 2024-04-20T21:39:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f9423400640 10 client.16146 handle_command_reply: tid=0 2024-04-20T21:39:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 client.16146 shutdown 2024-04-20T21:39:18.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 msgr2=0x564b058356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9434009610 tx=0x7f94340982e0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 shutdown_connections 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582fa90 0x564b05835be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b05848580 0x564b0584a970 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b05838670 0x564b05836120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 --2- 172.21.15.7:0/1480134397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058356a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 >> 172.21.15.7:0/1480134397 conn(0x564b058298a0 msgr2=0x564b0582b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f9423400640 0 client.16146 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 shutdown_connections 2024-04-20T21:39:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 1 -- 172.21.15.7:0/1480134397 wait complete. 2024-04-20T21:39:18.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.748+0000 7f944b98c740 20 client.16146 trim_cache size 0 max 16384 2024-04-20T21:39:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.763+0000 7f944b98c740 1 Processor -- start 2024-04-20T21:39:18.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 -- start start 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b058442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b05850210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b6940 con 0x564b05838670 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b4200 con 0x564b0582f170 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f944b98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b2050 con 0x564b0582fa90 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f9441e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f9442800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b05850210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f9441e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60534/0 (socket says 172.21.15.7:60534) 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f9441e00640 1 -- 172.21.15.7:0/3442537388 learned_addr learned my addr 172.21.15.7:0/3442537388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.764+0000 7f9441400640 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b058442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 -- 172.21.15.7:0/3442537388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 msgr2=0x564b058442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b058442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 -- 172.21.15.7:0/3442537388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 msgr2=0x564b05850210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b05850210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 -- 172.21.15.7:0/3442537388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b054a63f0 con 0x564b0582f170 2024-04-20T21:39:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f9441e00640 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94340d3d10 tx=0x7f9434005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3653ee32a96b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f942e600640 1 -- 172.21.15.7:0/3442537388 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434020070 con 0x564b0582f170 2024-04-20T21:39:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f942e600640 1 -- 172.21.15.7:0/3442537388 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943409c640 con 0x564b0582f170 2024-04-20T21:39:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.765+0000 7f942e600640 1 -- 172.21.15.7:0/3442537388 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340cdfc0 con 0x564b0582f170 2024-04-20T21:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- 172.21.15.7:0/3442537388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 msgr2=0x564b05837dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94340d3d10 tx=0x7f9434005ad0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- 172.21.15.7:0/3442537388 shutdown_connections 2024-04-20T21:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05838670 0x564b05850210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b058442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 --2- 172.21.15.7:0/3442537388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b0582f170 0x564b05837dd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- 172.21.15.7:0/3442537388 >> 172.21.15.7:0/3442537388 conn(0x564b058298a0 msgr2=0x564b05828960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- 172.21.15.7:0/3442537388 shutdown_connections 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- 172.21.15.7:0/3442537388 wait complete. 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.766+0000 7f944b98c740 1 -- start start 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 Processor -- start 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05832a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b05838670 0x564b05832f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b26c0 con 0x564b0582f170 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b3b30 con 0x564b05838670 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94441b2d90 con 0x564b0582fa90 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f944b98c740 10 client.0 _ll_register_callbacks cb 0x7f94340d3da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f9441e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f9441e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50664/0 (socket says 172.21.15.7:50664) 2024-04-20T21:39:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f9441e00640 1 -- 172.21.15.7:0/2498584873 learned_addr learned my addr 172.21.15.7:0/2498584873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.767+0000 7f9441400640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05832a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9442800640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b05838670 0x564b05832f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 -- 172.21.15.7:0/2498584873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 msgr2=0x564b05832a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05832a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 -- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b05838670 msgr2=0x564b05832f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b05838670 0x564b05832f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 -- 172.21.15.7:0/2498584873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b055707d0 con 0x564b0582f170 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9441e00640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9434005d00 tx=0x7f94340982e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b09e361e0a2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9423400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434020070 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f943409caa0 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.768+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943409c640 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94340ba050 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 10 client.16152 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b058aec80 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943409c640 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f94340d0110 con 0x564b0582f170 2024-04-20T21:39:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 20 client.16152 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b058af100 0x564b058b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 4 client.16152 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.769+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b0579b8e0 con 0x564b058af100 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f9441400640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b058af100 0x564b058b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f9441400640 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b058af100 0x564b058b14f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e1289e82df7f5e server_cookie=6ca0534070f3c9d0 in_seq=0 out_seq=0 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f9423400640 10 client.16152 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f9423400640 1 -- 172.21.15.7:0/2498584873 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b0579b8e0 con 0x564b058af100 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f9423400640 10 client.16152 handle_command_reply: tid=0 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 client.16152 shutdown 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 msgr2=0x564b058324c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9434005d00 tx=0x7f94340982e0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 shutdown_connections 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b05838670 0x564b05832f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564b058af100 0x564b058b14f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.770+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b0582fa90 0x564b05832a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f944b98c740 1 --2- 172.21.15.7:0/2498584873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b0582f170 0x564b058324c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f9423400640 0 client.16152 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 >> 172.21.15.7:0/2498584873 conn(0x564b058298a0 msgr2=0x564b058280d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 shutdown_connections 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f944b98c740 1 -- 172.21.15.7:0/2498584873 wait complete. 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f944b98c740 20 client.16152 trim_cache size 0 max 16384 2024-04-20T21:39:18.777 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 msgr2=0x7f94280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 0x7f94280a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f944406b2c0 tx=0x7f943c009600 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 msgr2=0x7f94441a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.771+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 0x7f94441a4a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9434005fd0 tx=0x7f943400dde0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 shutdown_connections 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94280a3da0 0x7f94280a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9444076db0 0x7f94441a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9444112be0 0x7f944419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 --2- 172.21.15.7:0/619857608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9444076490 0x7f94441a4a50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 >> 172.21.15.7:0/619857608 conn(0x7f94440ffa30 msgr2=0x7f94441143e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:18.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 shutdown_connections 2024-04-20T21:39:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:18.772+0000 7f9443200640 1 -- 172.21.15.7:0/619857608 wait complete. 2024-04-20T21:39:19.789 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:39:19.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:39:19.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:39:20.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 -- 172.21.15.7:0/3763755905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 msgr2=0x7f6bf410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 --2- 172.21.15.7:0/3763755905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf410e7c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009cd0 tx=0x7f6be402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 -- 172.21.15.7:0/3763755905 shutdown_connections 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 --2- 172.21.15.7:0/3763755905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf4108e40 0x7f6bf4115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 --2- 172.21.15.7:0/3763755905 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf41084a0 0x7f6bf4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 --2- 172.21.15.7:0/3763755905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf410e7c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.022+0000 7f6bfa200640 1 -- 172.21.15.7:0/3763755905 >> 172.21.15.7:0/3763755905 conn(0x7f6bf4077d70 msgr2=0x7f6bf4106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.023+0000 7f6bfa200640 1 -- 172.21.15.7:0/3763755905 shutdown_connections 2024-04-20T21:39:20.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.023+0000 7f6bfa200640 1 -- 172.21.15.7:0/3763755905 wait complete. 2024-04-20T21:39:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 Processor -- start 2024-04-20T21:39:20.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 -- start start 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 0x7f6bf41a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf4108e40 0x7f6bf41a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41186f0 con 0x7f6bf410e3e0 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4118570 con 0x7f6bf41084a0 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bfa200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf4118870 con 0x7f6bf4108e40 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.024+0000 7f6bf9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bebe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf4108e40 0x7f6bf41a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50688/0 (socket says 172.21.15.7:50688) 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf9800640 1 -- 172.21.15.7:0/2659792893 learned_addr learned my addr 172.21.15.7:0/2659792893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 0x7f6bf41a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2659792893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf4108e40 msgr2=0x7f6bf41a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf4108e40 0x7f6bf41a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 msgr2=0x7f6bf419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2659792893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf419f210 con 0x7f6bf41084a0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf9800640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.025+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 0x7f6bf41a48e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009bf0 tx=0x7f6be4032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.026+0000 7f6bf2000640 1 -- 172.21.15.7:0/2659792893 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4041020 con 0x7f6bf41084a0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.026+0000 7f6bf2000640 1 -- 172.21.15.7:0/2659792893 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be4034cd0 con 0x7f6bf41084a0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.026+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf419f4a0 con 0x7f6bf41084a0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.026+0000 7f6bf2000640 1 -- 172.21.15.7:0/2659792893 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4039040 con 0x7f6bf41084a0 2024-04-20T21:39:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.026+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf41ab7a0 con 0x7f6bf41084a0 2024-04-20T21:39:20.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.029+0000 7f6bfe5e3740 1 Processor -- start 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.029+0000 7f6bfe5e3740 1 -- start start 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.029+0000 7f6bf2000640 1 -- 172.21.15.7:0/2659792893 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6be4044020 con 0x7f6bf41084a0 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.029+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c121854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1218e5f0 0x560c121909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560c120e9d20 con 0x560c121850f0 2024-04-20T21:39:20.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560c120903e0 con 0x560c12185a10 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c120e5ce0 con 0x560c1218e5f0 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60576/0 (socket says 172.21.15.7:60576) 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c121854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 -- 172.21.15.7:0/4106870092 learned_addr learned my addr 172.21.15.7:0/4106870092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bebe00640 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1218e5f0 0x560c121909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 -- 172.21.15.7:0/4106870092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1218e5f0 msgr2=0x560c121909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1218e5f0 0x560c121909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 -- 172.21.15.7:0/4106870092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 msgr2=0x560c121854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c121854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.030+0000 7f6bf8e00640 1 -- 172.21.15.7:0/4106870092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c120af980 con 0x560c12185a10 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.031+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6be40efba0 tx=0x7f6be40fa4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5daaee058b37466a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.031+0000 7f6bea000640 1 -- 172.21.15.7:0/4106870092 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40f42d0 con 0x560c12185a10 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.031+0000 7f6bea000640 1 -- 172.21.15.7:0/4106870092 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be40ff030 con 0x560c12185a10 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.031+0000 7f6bea000640 1 -- 172.21.15.7:0/4106870092 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be4104880 con 0x560c12185a10 2024-04-20T21:39:20.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.031+0000 7f6bf2000640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 0x7f6bd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bf2000640 1 -- 172.21.15.7:0/2659792893 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6be40f2350 con 0x7f6bf41084a0 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bf9800640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 0x7f6bd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4106870092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 msgr2=0x560c1218e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6be40efba0 tx=0x7f6be40fa4b0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4106870092 shutdown_connections 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c12185a10 0x560c1218e0b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c1218e5f0 0x560c121909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4106870092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c121854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4106870092 >> 172.21.15.7:0/4106870092 conn(0x560c1217f820 msgr2=0x560c1217fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4106870092 shutdown_connections 2024-04-20T21:39:20.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4106870092 wait complete. 2024-04-20T21:39:20.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.032+0000 7f6bf9800640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 0x7f6bd80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009640 tx=0x7f6bec005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.033+0000 7f6bfe5e3740 1 -- start start 2024-04-20T21:39:20.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.033+0000 7f6bfe5e3740 1 Processor -- start 2024-04-20T21:39:20.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.033+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.033+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c1218bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c1218c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560c121911a0 con 0x560c121850f0 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560c120a37a0 con 0x560c1218e5f0 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560c12191320 con 0x560c12185a10 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bfe5e3740 10 client.0 _ll_register_callbacks cb 0x560c120a5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50700/0 (socket says 172.21.15.7:50700) 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2424289628 learned_addr learned my addr 172.21.15.7:0/2424289628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf9800640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c1218c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bebe00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c1218bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2424289628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 msgr2=0x560c1218bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c1218bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 msgr2=0x560c1218c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c1218c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bf8e00640 1 -- 172.21.15.7:0/2424289628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c11ec47e0 con 0x560c121850f0 2024-04-20T21:39:20.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.034+0000 7f6bebe00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c1218bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6be40be150 tx=0x7f6be4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ef2e6a51e5fc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6be2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40ba6f0 con 0x560c121850f0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6be4003970 con 0x560c121850f0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6be40ff030 con 0x560c121850f0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6be40bddb0 con 0x560c121850f0 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6bfe5e3740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:20.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c11f24f70 con 0x560c121850f0 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.035+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6be40ff030 con 0x560c121850f0 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f6be403c030 con 0x560c121850f0 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 20 client.16164 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c1219e4f0 0x560c121a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 4 client.16164 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:20.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c120f7740 con 0x560c1219e4f0 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.036+0000 7f6bebe00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c1219e4f0 0x560c121a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.037+0000 7f6bebe00640 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c1219e4f0 0x560c121a08e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2dbc2a6c847f1e server_cookie=c2d0b61e1d8956fe in_seq=0 out_seq=0 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.037+0000 7f6be2a00640 10 client.16164 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6be2a00640 1 -- 172.21.15.7:0/2424289628 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c120f7740 con 0x560c1219e4f0 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6be2a00640 10 client.16164 handle_command_reply: tid=0 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 client.16164 shutdown 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 msgr2=0x560c1218b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6be40be150 tx=0x7f6be4000f40 comp rx=0 tx=0).stop 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 shutdown_connections 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c1218c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c1219e4f0 0x560c121a08e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c1218bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/2424289628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c121850f0 0x560c1218b5f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6be2a00640 0 client.16164 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 >> 172.21.15.7:0/2424289628 conn(0x560c1217f820 msgr2=0x560c12181570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 shutdown_connections 2024-04-20T21:39:20.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/2424289628 wait complete. 2024-04-20T21:39:20.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.038+0000 7f6bfe5e3740 20 client.16164 trim_cache size 0 max 16384 2024-04-20T21:39:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.058+0000 7f6bfe5e3740 1 Processor -- start 2024-04-20T21:39:20.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.058+0000 7f6bfe5e3740 1 -- start start 2024-04-20T21:39:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.058+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c121850f0 0x560c12199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c121a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b6870 con 0x560c12185a10 2024-04-20T21:39:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b4130 con 0x560c1218e5f0 2024-04-20T21:39:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b1f80 con 0x560c121850f0 2024-04-20T21:39:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50714/0 (socket says 172.21.15.7:50714) 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 -- 172.21.15.7:0/4157734054 learned_addr learned my addr 172.21.15.7:0/4157734054 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bf9800640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c121a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c121850f0 0x560c12199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 -- 172.21.15.7:0/4157734054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c121850f0 msgr2=0x560c12199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c121850f0 0x560c12199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 -- 172.21.15.7:0/4157734054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 msgr2=0x560c121a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c121a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bebe00640 1 -- 172.21.15.7:0/4157734054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c11dbc170 con 0x560c12185a10 2024-04-20T21:39:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.059+0000 7f6bf9800640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c121a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:20.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.060+0000 7f6bebe00640 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005d00 tx=0x7f6bdc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bfb2d1c1ee4063 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.060+0000 7f6be9600640 1 -- 172.21.15.7:0/4157734054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc005220 con 0x560c12185a10 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.060+0000 7f6be9600640 1 -- 172.21.15.7:0/4157734054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc023030 con 0x560c12185a10 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.060+0000 7f6be9600640 1 -- 172.21.15.7:0/4157734054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc01f670 con 0x560c12185a10 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4157734054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 msgr2=0x560c1218bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005d00 tx=0x7f6bdc00b410 comp rx=0 tx=0).stop 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4157734054 shutdown_connections 2024-04-20T21:39:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c1218e5f0 0x560c121a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c12185a10 0x560c1218bcc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/4157734054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c121850f0 0x560c12199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4157734054 >> 172.21.15.7:0/4157734054 conn(0x560c1217f820 msgr2=0x560c1218b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4157734054 shutdown_connections 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/4157734054 wait complete. 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.061+0000 7f6bfe5e3740 1 -- start start 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 Processor -- start 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c12192a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 0x560c12192f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b25f0 con 0x560c1218e5f0 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b3a60 con 0x560c121850f0 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf41b2cc0 con 0x560c12185a10 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bfe5e3740 10 client.0 _ll_register_callbacks cb 0x7f6bec009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bf9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 0x560c12192f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bf8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60598/0 (socket says 172.21.15.7:60598) 2024-04-20T21:39:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bf8e00640 1 -- 172.21.15.7:0/3117094836 learned_addr learned my addr 172.21.15.7:0/3117094836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.062+0000 7f6bebe00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c12192a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 -- 172.21.15.7:0/3117094836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 msgr2=0x560c12192a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c12192a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 -- 172.21.15.7:0/3117094836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 msgr2=0x560c12192f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 0x560c12192f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 -- 172.21.15.7:0/3117094836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c121935b0 con 0x560c121850f0 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf9800640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 0x560c12192f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bf8e00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6be4004af0 tx=0x7f6be4005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7923fb9d6e2a1888 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6be2a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be40bdf90 con 0x560c121850f0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bdc023030 con 0x560c121850f0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.063+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f6be40c6c30 con 0x560c121850f0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6be40be220 con 0x560c121850f0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 10 client.26378 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c121935b0 con 0x560c121850f0 2024-04-20T21:39:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bdc023030 con 0x560c121850f0 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f6be40be6b0 con 0x560c121850f0 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 10 client.26378 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 20 client.26378 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c12205b50 0x560c12207f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 4 client.26378 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.064+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560c120f15b0 con 0x560c12205b50 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.065+0000 7f6bebe00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c12205b50 0x560c12207f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.065+0000 7f6bebe00640 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c12205b50 0x560c12207f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e61e6acc1584a5c server_cookie=70ad0eb820902502 in_seq=0 out_seq=0 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.065+0000 7f6be2a00640 10 client.26378 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6be2a00640 1 -- 172.21.15.7:0/3117094836 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560c120f15b0 con 0x560c12205b50 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6be2a00640 10 client.26378 handle_command_reply: tid=0 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 client.26378 shutdown 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 msgr2=0x560c121966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6be4004af0 tx=0x7f6be4005d80 comp rx=0 tx=0).stop 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 shutdown_connections 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560c12205b50 0x560c12207f40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560c1218e5f0 0x560c12192f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560c12185a10 0x560c12192a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 --2- 172.21.15.7:0/3117094836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560c121850f0 0x560c121966b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6be2a00640 0 client.26378 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 >> 172.21.15.7:0/3117094836 conn(0x560c1217f820 msgr2=0x560c1218be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 shutdown_connections 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 1 -- 172.21.15.7:0/3117094836 wait complete. 2024-04-20T21:39:20.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.067+0000 7f6bfe5e3740 20 client.26378 trim_cache size 0 max 16384 2024-04-20T21:39:20.074 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:39:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "765dfbd9-e018-47d1-b464-69089d1120bb", 2024-04-20T21:39:20.074 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:39:20.074 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 msgr2=0x7f6bd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 0x7f6bd80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009640 tx=0x7f6bec005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 msgr2=0x7f6bf41a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 0x7f6bf41a48e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009bf0 tx=0x7f6be4032ea0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 shutdown_connections 2024-04-20T21:39:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf410e3e0 0x7f6bf419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6bf4108e40 0x7f6bf41a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6bd80a3d20 0x7f6bd80a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 --2- 172.21.15.7:0/2659792893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bf41084a0 0x7f6bf41a48e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 >> 172.21.15.7:0/2659792893 conn(0x7f6bf4077d70 msgr2=0x7f6bf4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 shutdown_connections 2024-04-20T21:39:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.068+0000 7f6bfa200640 1 -- 172.21.15.7:0/2659792893 wait complete. 2024-04-20T21:39:20.089 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:20.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 -- 172.21.15.7:0/1130160480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1041f0 msgr2=0x7fa8cc1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 --2- 172.21.15.7:0/1130160480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1045d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c002bc30 comp rx=0 tx=0).stop 2024-04-20T21:39:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 -- 172.21.15.7:0/1130160480 shutdown_connections 2024-04-20T21:39:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 --2- 172.21.15.7:0/1130160480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 --2- 172.21.15.7:0/1130160480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc104ba0 0x7fa8cc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 --2- 172.21.15.7:0/1130160480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1045d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.307+0000 7fa8d3000640 1 -- 172.21.15.7:0/1130160480 >> 172.21.15.7:0/1130160480 conn(0x7fa8cc0ffa50 msgr2=0x7fa8cc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.308+0000 7fa8d3000640 1 -- 172.21.15.7:0/1130160480 shutdown_connections 2024-04-20T21:39:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.308+0000 7fa8d3000640 1 -- 172.21.15.7:0/1130160480 wait complete. 2024-04-20T21:39:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.309+0000 7fa8d3000640 1 Processor -- start 2024-04-20T21:39:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.309+0000 7fa8d3000640 1 -- start start 2024-04-20T21:39:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc19a760 0x7fa8cc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1143c0 con 0x7fa8cc19a760 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114240 con 0x7fa8cc1092c0 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d3000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc114540 con 0x7fa8cc1041f0 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc19a760 0x7fa8cc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60614/0 (socket says 172.21.15.7:60614) 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 -- 172.21.15.7:0/142679213 learned_addr learned my addr 172.21.15.7:0/142679213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 -- 172.21.15.7:0/142679213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc1041f0 msgr2=0x7fa8cc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 -- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc19a760 msgr2=0x7fa8cc19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc19a760 0x7fa8cc19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.310+0000 7fa8d1200640 1 -- 172.21.15.7:0/142679213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc19b360 con 0x7fa8cc1092c0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8d1200640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9f0 tx=0x7fa8b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8ca000640 1 -- 172.21.15.7:0/142679213 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b800ccc0 con 0x7fa8cc1092c0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8ca000640 1 -- 172.21.15.7:0/142679213 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8005500 con 0x7fa8cc1092c0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc1a73f0 con 0x7fa8cc1092c0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8ca000640 1 -- 172.21.15.7:0/142679213 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8010690 con 0x7fa8cc1092c0 2024-04-20T21:39:20.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.311+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc1a77f0 con 0x7fa8cc1092c0 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.314+0000 7fa8d73f4740 1 Processor -- start 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 -- start start 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559122e17150 con 0x559122eba880 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559122dbd370 con 0x559122eb1ca0 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559122e13110 con 0x559122eb1380 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8ca000640 1 -- 172.21.15.7:0/142679213 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa8b80046c0 con 0x7fa8cc1092c0 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60630/0 (socket says 172.21.15.7:60630) 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d1c00640 1 -- 172.21.15.7:0/558719908 learned_addr learned my addr 172.21.15.7:0/558719908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.315+0000 7fa8d2600640 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1200640 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 -- 172.21.15.7:0/558719908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 msgr2=0x559122eb1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 -- 172.21.15.7:0/558719908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 msgr2=0x559122ebcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 -- 172.21.15.7:0/558719908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559122bf37d0 con 0x559122eb1ca0 2024-04-20T21:39:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.316+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c0005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f484d2c4a6620e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8bea00640 1 -- 172.21.15.7:0/558719908 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0045070 con 0x559122eb1ca0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8bea00640 1 -- 172.21.15.7:0/558719908 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c002c880 con 0x559122eb1ca0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8bea00640 1 -- 172.21.15.7:0/558719908 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004c20 con 0x559122eb1ca0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8ca000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 0x7fa8b40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8ca000640 1 -- 172.21.15.7:0/142679213 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa8b80d0ae0 con 0x7fa8cc1092c0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.317+0000 7fa8d2600640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 0x7fa8b40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- 172.21.15.7:0/558719908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 msgr2=0x559122eba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c0005be0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- 172.21.15.7:0/558719908 shutdown_connections 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122eba340 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/558719908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- 172.21.15.7:0/558719908 >> 172.21.15.7:0/558719908 conn(0x559122eabab0 msgr2=0x559122eabe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- 172.21.15.7:0/558719908 shutdown_connections 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d2600640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 0x7fa8b40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005fd0 tx=0x7fa8c4009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- 172.21.15.7:0/558719908 wait complete. 2024-04-20T21:39:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.318+0000 7fa8d73f4740 1 -- start start 2024-04-20T21:39:20.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 Processor -- start 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eb1ca0 0x559122eb7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eba880 0x559122eb8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559122ebd610 con 0x559122eb1ca0 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559122dd0740 con 0x559122eba880 2024-04-20T21:39:20.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559122ebd790 con 0x559122eb1380 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d73f4740 10 client.0 _ll_register_callbacks cb 0x559122dcbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.319+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44718/0 (socket says 172.21.15.7:44718) 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 -- 172.21.15.7:0/1266819242 learned_addr learned my addr 172.21.15.7:0/1266819242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1200640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eb1ca0 0x559122eb7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d2600640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eba880 0x559122eb8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 -- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eba880 msgr2=0x559122eb8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eba880 0x559122eb8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 -- 172.21.15.7:0/1266819242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eb1ca0 msgr2=0x559122eb7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eb1ca0 0x559122eb7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 -- 172.21.15.7:0/1266819242 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559122b293f0 con 0x559122eb1380 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0036040 tx=0x7fa8c00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3261e7ccae3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8b3400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.320+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0045040 con 0x559122eb1380 2024-04-20T21:39:20.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa8c00052d0 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c002c880 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8c00054d0 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8d73f4740 10 client.26347 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559122ae90f0 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.321+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c002c880 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fa8c004a450 con 0x559122eb1380 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 10 client.26347 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:20.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 20 client.26347 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122ecaab0 0x559122eccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 4 client.26347 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559122e24b70 con 0x559122ecaab0 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.322+0000 7fa8d1200640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122ecaab0 0x559122eccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.323+0000 7fa8d1200640 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122ecaab0 0x559122eccea0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75009826a0096ee3 server_cookie=550bc4b31b0fd293 in_seq=0 out_seq=0 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.323+0000 7fa8b3400640 10 client.26347 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.323+0000 7fa8b3400640 1 -- 172.21.15.7:0/1266819242 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559122e24b70 con 0x559122ecaab0 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.323+0000 7fa8b3400640 10 client.26347 handle_command_reply: tid=0 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.323+0000 7fa8d73f4740 1 client.26347 shutdown 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 msgr2=0x559122eb7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0036040 tx=0x7fa8c00046c0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 shutdown_connections 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eba880 0x559122eb8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122ecaab0 0x559122eccea0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eb1ca0 0x559122eb7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/1266819242 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122eb7900 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8b3400640 0 client.26347 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 >> 172.21.15.7:0/1266819242 conn(0x559122eabab0 msgr2=0x559122ead800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 shutdown_connections 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 1 -- 172.21.15.7:0/1266819242 wait complete. 2024-04-20T21:39:20.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.324+0000 7fa8d73f4740 20 client.26347 trim_cache size 0 max 16384 2024-04-20T21:39:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.343+0000 7fa8d73f4740 1 Processor -- start 2024-04-20T21:39:20.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.343+0000 7fa8d73f4740 1 -- start start 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1ca0 0x559122eb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ead560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1b2500 con 0x559122eba880 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1afdc0 con 0x559122eb1380 2024-04-20T21:39:20.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1adc30 con 0x559122eb1ca0 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60650/0 (socket says 172.21.15.7:60650) 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 -- 172.21.15.7:0/6586605 learned_addr learned my addr 172.21.15.7:0/6586605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1200640 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1ca0 0x559122eb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d2600640 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ead560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 -- 172.21.15.7:0/6586605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1ca0 msgr2=0x559122eb71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1ca0 0x559122eb71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 -- 172.21.15.7:0/6586605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 msgr2=0x559122ead560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ead560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.344+0000 7fa8d1c00640 1 -- 172.21.15.7:0/6586605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559122eab5e0 con 0x559122eb1380 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.345+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0001eb0 tx=0x7fa8c0041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3969805c152eecc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.345+0000 7fa8be000640 1 -- 172.21.15.7:0/6586605 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0045070 con 0x559122eb1380 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.345+0000 7fa8be000640 1 -- 172.21.15.7:0/6586605 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c002c880 con 0x559122eb1380 2024-04-20T21:39:20.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.345+0000 7fa8be000640 1 -- 172.21.15.7:0/6586605 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c005a030 con 0x559122eb1380 2024-04-20T21:39:20.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- 172.21.15.7:0/6586605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 msgr2=0x559122ec6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0001eb0 tx=0x7fa8c0041040 comp rx=0 tx=0).stop 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- 172.21.15.7:0/6586605 shutdown_connections 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ead560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1ca0 0x559122eb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/6586605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1380 0x559122ec6500 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- 172.21.15.7:0/6586605 >> 172.21.15.7:0/6586605 conn(0x559122eabab0 msgr2=0x559122eace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- 172.21.15.7:0/6586605 shutdown_connections 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- 172.21.15.7:0/6586605 wait complete. 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.346+0000 7fa8d73f4740 1 -- start start 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 Processor -- start 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122ec2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ae280 con 0x559122eba880 2024-04-20T21:39:20.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1af6f0 con 0x559122eb1ca0 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc1ae950 con 0x559122eb1380 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d73f4740 10 client.0 _ll_register_callbacks cb 0x7fa8c4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:60664/0 (socket says 172.21.15.7:60664) 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.347+0000 7fa8d1200640 1 -- 172.21.15.7:0/2180060568 learned_addr learned my addr 172.21.15.7:0/2180060568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:20.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122ec2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 -- 172.21.15.7:0/2180060568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 msgr2=0x559122ec2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122ec2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 -- 172.21.15.7:0/2180060568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 msgr2=0x559122ebe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 -- 172.21.15.7:0/2180060568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559122ebec80 con 0x559122eb1ca0 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d1200640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80ce280 tx=0x7fa8b8087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509e2f513ff1bb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8b3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80bd0b0 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa8b80990e0 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.348+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c002c880 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8b809ca70 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 10 client.26396 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559122f30d40 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c002c880 con 0x559122eb1ca0 2024-04-20T21:39:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fa8b809ccd0 con 0x559122eb1ca0 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 10 client.26396 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 20 client.26396 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122f314f0 0x559122f33910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 4 client.26396 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559122e1e9e0 con 0x559122f314f0 2024-04-20T21:39:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.349+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122f314f0 0x559122f33910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d1c00640 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122f314f0 0x559122f33910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45b3eb8b37c4734 server_cookie=3cc9369f091de89c in_seq=0 out_seq=0 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8b3400640 10 client.26396 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8b3400640 1 -- 172.21.15.7:0/2180060568 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559122e1e9e0 con 0x559122f314f0 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8b3400640 10 client.26396 handle_command_reply: tid=0 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 client.26396 shutdown 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 msgr2=0x559122ebe050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80ce280 tx=0x7fa8b8087040 comp rx=0 tx=0).stop 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 shutdown_connections 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559122eb1ca0 0x559122ebe050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559122f314f0 0x559122f33910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559122eba880 0x559122ebe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 --2- 172.21.15.7:0/2180060568 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559122eb1380 0x559122ec2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8b3400640 0 client.26396 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.350+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 >> 172.21.15.7:0/2180060568 conn(0x559122eabab0 msgr2=0x559122eb81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 shutdown_connections 2024-04-20T21:39:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d73f4740 1 -- 172.21.15.7:0/2180060568 wait complete. 2024-04-20T21:39:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d73f4740 20 client.26396 trim_cache size 0 max 16384 2024-04-20T21:39:20.360 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:39:20.360 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "765dfbd9-e018-47d1-b464-69089d1120bb": { 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "765dfbd9-e018-47d1-b464-69089d1120bb", 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:39:20.361 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 msgr2=0x7fa8b40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 0x7fa8b40a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4005fd0 tx=0x7fa8c4009500 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 msgr2=0x7fa8cc1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e9f0 tx=0x7fa8b800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 shutdown_connections 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa8b40a3da0 0x7fa8b40a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8cc1092c0 0x7fa8cc1a0b90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8cc19a760 0x7fa8cc19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 --2- 172.21.15.7:0/142679213 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa8cc1041f0 0x7fa8cc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:20.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.351+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 >> 172.21.15.7:0/142679213 conn(0x7fa8cc0ffa50 msgr2=0x7fa8cc100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.352+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 shutdown_connections 2024-04-20T21:39:20.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:20.352+0000 7fa8d3000640 1 -- 172.21.15.7:0/142679213 wait complete. 2024-04-20T21:39:20.370 INFO:tasks.cephfs.filesystem:scrub status for tag:765dfbd9-e018-47d1-b464-69089d1120bb - {'path': '/', 'tag': '765dfbd9-e018-47d1-b464-69089d1120bb', 'options': 'recursive,force'} 2024-04-20T21:39:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:20 smithi007 ceph-mon[16867]: pgmap v183: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 8 op/s 2024-04-20T21:39:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:20 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:20 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:20 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:20.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:20 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:20.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:20 smithi016 ceph-mon[21409]: pgmap v183: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 8 op/s 2024-04-20T21:39:20.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:20 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:20.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:20 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:20.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:20 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:20.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:20 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:20 smithi186 ceph-mon[22157]: pgmap v183: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 8 op/s 2024-04-20T21:39:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:20 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:20 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:20.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:20 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:20.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:20 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:39:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:21 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:39:21.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:21.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:21 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:39:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:21.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:21 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:22 smithi016 ceph-mon[21409]: pgmap v184: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-04-20T21:39:22.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:22 smithi186 ceph-mon[22157]: pgmap v184: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-04-20T21:39:22.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:22 smithi007 ceph-mon[16867]: pgmap v184: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 5 op/s 2024-04-20T21:39:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:24 smithi016 ceph-mon[21409]: pgmap v185: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-04-20T21:39:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:24 smithi186 ceph-mon[22157]: pgmap v185: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-04-20T21:39:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:24 smithi007 ceph-mon[16867]: pgmap v185: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.2 KiB/s wr, 7 op/s 2024-04-20T21:39:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:26 smithi016 ceph-mon[21409]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-20T21:39:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:26 smithi186 ceph-mon[22157]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-20T21:39:26.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:26 smithi007 ceph-mon[16867]: pgmap v186: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-20T21:39:27.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:27 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:39:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:27 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:39:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:27 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:39:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:27 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:39:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:27 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:39:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:27 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:39:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:28 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:39:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:28 smithi016 ceph-mon[21409]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-20T21:39:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:28 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:39:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:28 smithi186 ceph-mon[22157]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-20T21:39:28.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:28 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:39:28.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:28 smithi007 ceph-mon[16867]: pgmap v187: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 5 op/s 2024-04-20T21:39:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:30 smithi016 ceph-mon[21409]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-20T21:39:30.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:30 smithi186 ceph-mon[22157]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-20T21:39:30.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:30 smithi007 ceph-mon[16867]: pgmap v188: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 8 op/s 2024-04-20T21:39:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:32 smithi016 ceph-mon[21409]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-04-20T21:39:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:32 smithi186 ceph-mon[22157]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-04-20T21:39:32.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:32 smithi007 ceph-mon[16867]: pgmap v189: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.0 KiB/s wr, 5 op/s 2024-04-20T21:39:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:34 smithi016 ceph-mon[21409]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:34 smithi186 ceph-mon[22157]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:34.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:34 smithi007 ceph-mon[16867]: pgmap v190: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:36 smithi016 ceph-mon[21409]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:36 smithi186 ceph-mon[22157]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:36 smithi007 ceph-mon[16867]: pgmap v191: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.0 KiB/s wr, 6 op/s 2024-04-20T21:39:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:38 smithi016 ceph-mon[21409]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:39:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:38 smithi186 ceph-mon[22157]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:39:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:38 smithi007 ceph-mon[16867]: pgmap v192: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:39:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:40 smithi016 ceph-mon[21409]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:39:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:40 smithi186 ceph-mon[22157]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:39:40.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:40 smithi007 ceph-mon[16867]: pgmap v193: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:39:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:42 smithi186 ceph-mon[22157]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:42 smithi007 ceph-mon[16867]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:42.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:42 smithi016 ceph-mon[21409]: pgmap v194: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:43.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:43 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:43.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:43 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:43.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:43 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:39:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:44 smithi186 ceph-mon[22157]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:44 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:44.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:44 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:44.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:44 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:39:44.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:44 smithi007 ceph-mon[16867]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:44 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:44 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:44 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:39:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:44 smithi016 ceph-mon[21409]: pgmap v195: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:44 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:39:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:44 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:39:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:44 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:39:46.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:46 smithi016 ceph-mon[21409]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:39:46.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:46 smithi007 ceph-mon[16867]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:39:46.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:46 smithi186 ceph-mon[22157]: pgmap v196: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:39:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:48 smithi007 ceph-mon[16867]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:48.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:48 smithi016 ceph-mon[21409]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:48.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:48 smithi186 ceph-mon[22157]: pgmap v197: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:50.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:50.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.598+0000 7f78ea000640 1 -- 172.21.15.7:0/4048151343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102b40 msgr2=0x7f78e4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.598+0000 7f78ea000640 1 --2- 172.21.15.7:0/4048151343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102b40 0x7f78e4106b80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009ca0 tx=0x7f78cc02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.598+0000 7f78ea000640 1 -- 172.21.15.7:0/4048151343 shutdown_connections 2024-04-20T21:39:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.598+0000 7f78ea000640 1 --2- 172.21.15.7:0/4048151343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 0x7f78e410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.598+0000 7f78ea000640 1 --2- 172.21.15.7:0/4048151343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102b40 0x7f78e4106b80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.599+0000 7f78ea000640 1 --2- 172.21.15.7:0/4048151343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4102190 0x7f78e4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.599+0000 7f78ea000640 1 -- 172.21.15.7:0/4048151343 >> 172.21.15.7:0/4048151343 conn(0x7f78e40fda10 msgr2=0x7f78e40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.599+0000 7f78ea000640 1 -- 172.21.15.7:0/4048151343 shutdown_connections 2024-04-20T21:39:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.599+0000 7f78ea000640 1 -- 172.21.15.7:0/4048151343 wait complete. 2024-04-20T21:39:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 Processor -- start 2024-04-20T21:39:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 -- start start 2024-04-20T21:39:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 0x7f78e40747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4074ce0 0x7f78e41a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41122f0 con 0x7f78e4102190 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4112170 con 0x7f78e4107260 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.600+0000 7f78ea000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4112470 con 0x7f78e4074ce0 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56482/0 (socket says 172.21.15.7:56482) 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 -- 172.21.15.7:0/3815925746 learned_addr learned my addr 172.21.15.7:0/3815925746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e2a00640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 0x7f78e40747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3e00640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4074ce0 0x7f78e41a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 -- 172.21.15.7:0/3815925746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4074ce0 msgr2=0x7f78e41a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4074ce0 0x7f78e41a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 -- 172.21.15.7:0/3815925746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 msgr2=0x7f78e40747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 0x7f78e40747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 -- 172.21.15.7:0/3815925746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e41a7160 con 0x7f78e4102190 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.601+0000 7f78e3400640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9f0 tx=0x7f78d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.602+0000 7f78dbe00640 1 -- 172.21.15.7:0/3815925746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d400cd80 con 0x7f78e4102190 2024-04-20T21:39:50.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.602+0000 7f78dbe00640 1 -- 172.21.15.7:0/3815925746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d4005500 con 0x7f78e4102190 2024-04-20T21:39:50.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.602+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41a7450 con 0x7f78e4102190 2024-04-20T21:39:50.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.602+0000 7f78dbe00640 1 -- 172.21.15.7:0/3815925746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4010690 con 0x7f78e4102190 2024-04-20T21:39:50.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.602+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e41a7920 con 0x7f78e4102190 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.605+0000 7f78dbe00640 1 -- 172.21.15.7:0/3815925746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f78d40046c0 con 0x7f78e4102190 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.605+0000 7f78ecd8d740 1 Processor -- start 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 -- start start 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 0x5637687d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563768738150 con 0x5637687d21a0 2024-04-20T21:39:50.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5637686de370 con 0x5637687d2ac0 2024-04-20T21:39:50.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.606+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563768734110 con 0x5637687db6a0 2024-04-20T21:39:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 0x5637687d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33818/0 (socket says 172.21.15.7:33818) 2024-04-20T21:39:50.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3400640 1 -- 172.21.15.7:0/2670860637 learned_addr learned my addr 172.21.15.7:0/2670860637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 -- 172.21.15.7:0/2670860637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 msgr2=0x5637687dda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687dda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 -- 172.21.15.7:0/2670860637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 msgr2=0x5637687db160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3e00640 1 -- 172.21.15.7:0/2670860637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637685147d0 con 0x5637687d21a0 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.607+0000 7f78e3400640 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78e3e00640 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 0x5637687d2580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009a70 tx=0x7f78dc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1adfda8451c0430 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78dbe00640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 0x7f78c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78dbe00640 1 -- 172.21.15.7:0/3815925746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f78d40d0c40 con 0x7f78e4102190 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78d3e00640 1 -- 172.21.15.7:0/2670860637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011d40 con 0x5637687d21a0 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78d3e00640 1 -- 172.21.15.7:0/2670860637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc012930 con 0x5637687d21a0 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78d3e00640 1 -- 172.21.15.7:0/2670860637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00fa10 con 0x5637687d21a0 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.608+0000 7f78e3400640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 0x7f78c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- 172.21.15.7:0/2670860637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 msgr2=0x5637687d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78e3400640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 0x7f78c40a6210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78d409cbc0 tx=0x7f78d40d4790 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 0x5637687d2580 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009a70 tx=0x7f78dc009f30 comp rx=0 tx=0).stop 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- 172.21.15.7:0/2670860637 shutdown_connections 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d2ac0 0x5637687db160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/2670860637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d21a0 0x5637687d2580 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- 172.21.15.7:0/2670860637 >> 172.21.15.7:0/2670860637 conn(0x5637687cc8d0 msgr2=0x5637687cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- 172.21.15.7:0/2670860637 shutdown_connections 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- 172.21.15.7:0/2670860637 wait complete. 2024-04-20T21:39:50.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.609+0000 7f78ecd8d740 1 -- start start 2024-04-20T21:39:50.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 Processor -- start 2024-04-20T21:39:50.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637687de490 con 0x5637687d2ac0 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5637686f1740 con 0x5637687db6a0 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5637687de610 con 0x5637687d21a0 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78ecd8d740 10 client.0 _ll_register_callbacks cb 0x5637686ecb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.610+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56494/0 (socket says 172.21.15.7:56494) 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 -- 172.21.15.7:0/46948378 learned_addr learned my addr 172.21.15.7:0/46948378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e3e00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e3e00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 -- 172.21.15.7:0/46948378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 msgr2=0x5637687d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687d8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 -- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 msgr2=0x5637687d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 -- 172.21.15.7:0/46948378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376844a3f0 con 0x5637687d2ac0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78e2a00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000f80 tx=0x7f78cc0323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c997240c9d2a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.611+0000 7f78d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc044070 con 0x5637687d2ac0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f78cc032790 con 0x5637687d2ac0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78dc012930 con 0x5637687d2ac0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78cc039040 con 0x5637687d2ac0 2024-04-20T21:39:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78ecd8d740 10 client.16200 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56376840a0f0 con 0x5637687d2ac0 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.612+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78dc012930 con 0x5637687d2ac0 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f78cc0044f0 con 0x5637687d2ac0 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 10 client.16200 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 20 client.16200 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637687eb9d0 0x5637687eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 4 client.16200 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563768745b70 con 0x5637687eb9d0 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.613+0000 7f78e3e00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637687eb9d0 0x5637687eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.614+0000 7f78e3e00640 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637687eb9d0 0x5637687eddf0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701a68fa6b5c4325 server_cookie=3190f0587ee37939 in_seq=0 out_seq=0 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.614+0000 7f78d0c00640 10 client.16200 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78d0c00640 1 -- 172.21.15.7:0/46948378 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563768745b70 con 0x5637687eb9d0 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78d0c00640 10 client.16200 handle_command_reply: tid=0 2024-04-20T21:39:50.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 client.16200 shutdown 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 msgr2=0x5637687d8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000f80 tx=0x7f78cc0323d0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 shutdown_connections 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637687eb9d0 0x5637687eddf0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8c80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/46948378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78d0c00640 0 client.16200 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 >> 172.21.15.7:0/46948378 conn(0x5637687cc8d0 msgr2=0x5637687ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 shutdown_connections 2024-04-20T21:39:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 1 -- 172.21.15.7:0/46948378 wait complete. 2024-04-20T21:39:50.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.615+0000 7f78ecd8d740 20 client.16200 trim_cache size 0 max 16384 2024-04-20T21:39:50.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.634+0000 7f78ecd8d740 1 Processor -- start 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 -- start start 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687e7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41b26b0 con 0x5637687d2ac0 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41aff70 con 0x5637687db6a0 2024-04-20T21:39:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41adde0 con 0x5637687d21a0 2024-04-20T21:39:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56510/0 (socket says 172.21.15.7:56510) 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.635+0000 7f78e2a00640 1 -- 172.21.15.7:0/1662181484 learned_addr learned my addr 172.21.15.7:0/1662181484 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e3e00640 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 -- 172.21.15.7:0/1662181484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 msgr2=0x5637687e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 -- 172.21.15.7:0/1662181484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 msgr2=0x5637687cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 -- 172.21.15.7:0/1662181484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637687d28f0 con 0x5637687d2ac0 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e3e00640 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.636+0000 7f78e2a00640 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c130 tx=0x7f78cc0527d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f9233c850cca1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78d3400640 1 -- 172.21.15.7:0/1662181484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc057070 con 0x5637687d2ac0 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78d3400640 1 -- 172.21.15.7:0/1662181484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc052970 con 0x5637687d2ac0 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78d3400640 1 -- 172.21.15.7:0/1662181484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc044070 con 0x5637687d2ac0 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 -- 172.21.15.7:0/1662181484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 msgr2=0x5637687d8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f78cc02c130 tx=0x7f78cc0527d0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 -- 172.21.15.7:0/1662181484 shutdown_connections 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687db6a0 0x5637687cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687d8fe0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/1662181484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687d21a0 0x5637687e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 -- 172.21.15.7:0/1662181484 >> 172.21.15.7:0/1662181484 conn(0x5637687cc8d0 msgr2=0x5637687cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 -- 172.21.15.7:0/1662181484 shutdown_connections 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.637+0000 7f78ecd8d740 1 -- 172.21.15.7:0/1662181484 wait complete. 2024-04-20T21:39:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.638+0000 7f78ecd8d740 1 -- start start 2024-04-20T21:39:50.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.638+0000 7f78ecd8d740 1 Processor -- start 2024-04-20T21:39:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.638+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.638+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78ecd8d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41ae430 con 0x5637687d2ac0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41af8a0 con 0x5637687d21a0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78ecd8d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41aeb00 con 0x5637687db6a0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78ecd8d740 10 client.0 _ll_register_callbacks cb 0x7f78d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33848/0 (socket says 172.21.15.7:33848) 2024-04-20T21:39:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 -- 172.21.15.7:0/447503908 learned_addr learned my addr 172.21.15.7:0/447503908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 -- 172.21.15.7:0/447503908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 msgr2=0x5637687df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 -- 172.21.15.7:0/447503908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 msgr2=0x5637687df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 -- 172.21.15.7:0/447503908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563768850f30 con 0x5637687d21a0 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78e3e00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009a10 tx=0x7f78cc009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a27da74444ecd985 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.639+0000 7f78d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00fd40 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78cc052970 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f78dc005040 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78dc005240 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78ecd8d740 10 client.26414 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637688520c0 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.640+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78cc052970 con 0x5637687d21a0 2024-04-20T21:39:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f78dc03b030 con 0x5637687d21a0 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 10 client.26414 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 20 client.26414 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637688526d0 0x563768854af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 4 client.26414 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56376873f9e0 con 0x5637688526d0 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.641+0000 7f78e2a00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637688526d0 0x563768854af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78e2a00640 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637688526d0 0x563768854af0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed40476a79606ed server_cookie=2ac6324649050968 in_seq=0 out_seq=0 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78d0c00640 10 client.26414 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78d0c00640 1 -- 172.21.15.7:0/447503908 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56376873f9e0 con 0x5637688526d0 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78d0c00640 10 client.26414 handle_command_reply: tid=0 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 client.26414 shutdown 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 msgr2=0x5637687e3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f78dc009a10 tx=0x7f78cc009930 comp rx=0 tx=0).stop 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 shutdown_connections 2024-04-20T21:39:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5637688526d0 0x563768854af0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637687db6a0 0x5637687df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637687d2ac0 0x5637687df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 --2- 172.21.15.7:0/447503908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5637687d21a0 0x5637687e3d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 >> 172.21.15.7:0/447503908 conn(0x5637687cc8d0 msgr2=0x5637687cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78d0c00640 0 client.26414 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 shutdown_connections 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 1 -- 172.21.15.7:0/447503908 wait complete. 2024-04-20T21:39:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.642+0000 7f78ecd8d740 20 client.26414 trim_cache size 0 max 16384 2024-04-20T21:39:50.651 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:39:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:39:50.651 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 msgr2=0x7f78c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 0x7f78c40a6210 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78d409cbc0 tx=0x7f78d40d4790 comp rx=0 tx=0).stop 2024-04-20T21:39:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 msgr2=0x7f78e4074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9f0 tx=0x7f78d400edf0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 shutdown_connections 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f78c40a3d50 0x7f78c40a6210 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e4107260 0x7f78e40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78e4074ce0 0x7f78e41a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 --2- 172.21.15.7:0/3815925746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78e4102190 0x7f78e4074260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 >> 172.21.15.7:0/3815925746 conn(0x7f78e40fda10 msgr2=0x7f78e40fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 shutdown_connections 2024-04-20T21:39:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.643+0000 7f78ea000640 1 -- 172.21.15.7:0/3815925746 wait complete. 2024-04-20T21:39:50.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:39:50.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:39:50.740 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:50 smithi007 ceph-mon[16867]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:39:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 -- 172.21.15.7:0/3358752621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea801084a0 msgr2=0x7fea80108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 --2- 172.21.15.7:0/3358752621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea801084a0 0x7fea80108900 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009860 tx=0x7fea6c02bc10 comp rx=0 tx=0).stop 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 -- 172.21.15.7:0/3358752621 shutdown_connections 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 --2- 172.21.15.7:0/3358752621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 --2- 172.21.15.7:0/3358752621 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea801084a0 0x7fea80108900 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 --2- 172.21.15.7:0/3358752621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8010e500 0x7fea8010e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 -- 172.21.15.7:0/3358752621 >> 172.21.15.7:0/3358752621 conn(0x7fea80077d30 msgr2=0x7fea80106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.880+0000 7fea7f200640 1 -- 172.21.15.7:0/3358752621 shutdown_connections 2024-04-20T21:39:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.881+0000 7fea7f200640 1 -- 172.21.15.7:0/3358752621 wait complete. 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 Processor -- start 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 -- start start 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 0x7fea801a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea8010e500 0x7fea8019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fea801187f0 con 0x7fea801084a0 2024-04-20T21:39:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea80118670 con 0x7fea80108e40 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.882+0000 7fea7f200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fea80118970 con 0x7fea8010e500 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 0x7fea801a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 0x7fea801a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56538/0 (socket says 172.21.15.7:56538) 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33866/0 (socket says 172.21.15.7:33866) 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7de00640 1 -- 172.21.15.7:0/208953081 learned_addr learned my addr 172.21.15.7:0/208953081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7e800640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea8010e500 0x7fea8019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 -- 172.21.15.7:0/208953081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea8010e500 msgr2=0x7fea8019eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea8010e500 0x7fea8019eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 -- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 msgr2=0x7fea801a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 0x7fea801a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.883+0000 7fea7d400640 1 -- 172.21.15.7:0/208953081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8019f3f0 con 0x7fea80108e40 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea7d400640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea6c02c150 tx=0x7fea6c0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c041020 con 0x7fea80108e40 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8019f680 con 0x7fea80108e40 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea6c0044d0 con 0x7fea80108e40 2024-04-20T21:39:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c039040 con 0x7fea80108e40 2024-04-20T21:39:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.884+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea801ab810 con 0x7fea80108e40 2024-04-20T21:39:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.885+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea80109b90 con 0x7fea80108e40 2024-04-20T21:39:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.888+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fea6c044020 con 0x7fea80108e40 2024-04-20T21:39:50.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.889+0000 7fea76a00640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 0x7fea640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.889+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fea6c0f2250 con 0x7fea80108e40 2024-04-20T21:39:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.889+0000 7fea7de00640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 0x7fea640a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.890+0000 7fea7de00640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 0x7fea640a61d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fea700058c0 tx=0x7fea70009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:50.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:50.892+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fea6c04b020 con 0x7fea80108e40 2024-04-20T21:39:50.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:50 smithi016 ceph-mon[21409]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:39:50.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:50 smithi186 ceph-mon[22157]: pgmap v198: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:39:51.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.140+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea8004fd60 con 0x7fea80108e40 2024-04-20T21:39:51.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.141+0000 7fea76a00640 1 -- 172.21.15.7:0/208953081 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7fea8004fd60 con 0x7fea80108e40 2024-04-20T21:39:51.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:39:51.142 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:39:51.142 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:39:51.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.144+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 msgr2=0x7fea640a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.144+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 0x7fea640a61d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fea700058c0 tx=0x7fea70009260 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.144+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 msgr2=0x7fea801a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.144+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fea6c02c150 tx=0x7fea6c0346f0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 shutdown_connections 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fea640a3d10 0x7fea640a61d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea80108e40 0x7fea801a4fb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fea8010e500 0x7fea8019eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 --2- 172.21.15.7:0/208953081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea801084a0 0x7fea801a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 >> 172.21.15.7:0/208953081 conn(0x7fea80077d30 msgr2=0x7fea80078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 shutdown_connections 2024-04-20T21:39:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.145+0000 7fea7f200640 1 -- 172.21.15.7:0/208953081 wait complete. 2024-04-20T21:39:51.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:39:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.373+0000 7f6104c00640 1 -- 172.21.15.7:0/1812694162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61001043a0 msgr2=0x7f6100104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.373+0000 7f6104c00640 1 --2- 172.21.15.7:0/1812694162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61001043a0 0x7f6100104780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009900 tx=0x7f60f002bc30 comp rx=0 tx=0).stop 2024-04-20T21:39:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 -- 172.21.15.7:0/1812694162 shutdown_connections 2024-04-20T21:39:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 --2- 172.21.15.7:0/1812694162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6100104d50 0x7f6100108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 --2- 172.21.15.7:0/1812694162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f6100111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 --2- 172.21.15.7:0/1812694162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61001043a0 0x7f6100104780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 -- 172.21.15.7:0/1812694162 >> 172.21.15.7:0/1812694162 conn(0x7f61000ffc20 msgr2=0x7f6100102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 -- 172.21.15.7:0/1812694162 shutdown_connections 2024-04-20T21:39:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.374+0000 7f6104c00640 1 -- 172.21.15.7:0/1812694162 wait complete. 2024-04-20T21:39:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 Processor -- start 2024-04-20T21:39:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 -- start start 2024-04-20T21:39:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61001043a0 0x7f61001a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610019a8e0 0x7f610019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114500 con 0x7f6100109470 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114380 con 0x7f61001043a0 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.375+0000 7f6104c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6100114680 con 0x7f610019a8e0 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61001043a0 0x7f61001a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56546/0 (socket says 172.21.15.7:56546) 2024-04-20T21:39:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 -- 172.21.15.7:0/3790960860 learned_addr learned my addr 172.21.15.7:0/3790960860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60fe800640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610019a8e0 0x7f610019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 -- 172.21.15.7:0/3790960860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610019a8e0 msgr2=0x7f610019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610019a8e0 0x7f610019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 -- 172.21.15.7:0/3790960860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61001043a0 msgr2=0x7f61001a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61001043a0 0x7f61001a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 -- 172.21.15.7:0/3790960860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610019b400 con 0x7f6100109470 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.376+0000 7f60f7e00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d770 tx=0x7f60e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.377+0000 7f60f6a00640 1 -- 172.21.15.7:0/3790960860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8017070 con 0x7f6100109470 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.377+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610019b6f0 con 0x7f6100109470 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.377+0000 7f60f6a00640 1 -- 172.21.15.7:0/3790960860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e8009c30 con 0x7f6100109470 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.377+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61001a78f0 con 0x7f6100109470 2024-04-20T21:39:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.377+0000 7f60f6a00640 1 -- 172.21.15.7:0/3790960860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8021060 con 0x7f6100109470 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 Processor -- start 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f60f6a00640 1 -- 172.21.15.7:0/3790960860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f60e801a070 con 0x7f6100109470 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 -- start start 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14952f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b149558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b148af050 con 0x564b14949fb0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b14855490 con 0x564b1494a8d0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.381+0000 7f6107988740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564b148ab010 con 0x564b149534b0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14952f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56552/0 (socket says 172.21.15.7:56552) 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fe800640 1 -- 172.21.15.7:0/1853376637 learned_addr learned my addr 172.21.15.7:0/1853376637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b149558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 -- 172.21.15.7:0/1853376637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 msgr2=0x564b149558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b149558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 -- 172.21.15.7:0/1853376637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 msgr2=0x564b1494a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fde00640 1 -- 172.21.15.7:0/1853376637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1485d070 con 0x564b1494a8d0 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b149558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.382+0000 7f60fe800640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60fde00640 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14952f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f60f0000c00 tx=0x7f60f0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76006eb05500066 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60ef000640 1 -- 172.21.15.7:0/1853376637 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0042070 con 0x564b1494a8d0 2024-04-20T21:39:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60f6a00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 0x7f60e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60ef000640 1 -- 172.21.15.7:0/1853376637 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f002ca80 con 0x564b1494a8d0 2024-04-20T21:39:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60ef000640 1 -- 172.21.15.7:0/1853376637 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0004f00 con 0x564b1494a8d0 2024-04-20T21:39:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60f6a00640 1 -- 172.21.15.7:0/3790960860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f60e80d0350 con 0x7f6100109470 2024-04-20T21:39:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.383+0000 7f60fe800640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 0x7f60e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 -- 172.21.15.7:0/1853376637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 msgr2=0x564b14952f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14952f70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f60f0000c00 tx=0x7f60f0005e80 comp rx=0 tx=0).stop 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 -- 172.21.15.7:0/1853376637 shutdown_connections 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14952f70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b149558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 --2- 172.21.15.7:0/1853376637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b1494a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 -- 172.21.15.7:0/1853376637 >> 172.21.15.7:0/1853376637 conn(0x564b149446e0 msgr2=0x564b14944ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f60fe800640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 0x7f60e40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005f40 tx=0x7f60f80096a0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 -- 172.21.15.7:0/1853376637 shutdown_connections 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.384+0000 7f6107988740 1 -- 172.21.15.7:0/1853376637 wait complete. 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 -- start start 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 Processor -- start 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14950a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b14950fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b14956240 con 0x564b14949fb0 2024-04-20T21:39:51.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b14868860 con 0x564b1494a8d0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564b149563c0 con 0x564b149534b0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.385+0000 7f6107988740 10 client.0 _ll_register_callbacks cb 0x564b14836fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56558/0 (socket says 172.21.15.7:56558) 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 -- 172.21.15.7:0/1531611749 learned_addr learned my addr 172.21.15.7:0/1531611749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fe800640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b14950fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14950a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 -- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 msgr2=0x564b14950fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b14950fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 -- 172.21.15.7:0/1531611749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 msgr2=0x564b14950a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14950a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 -- 172.21.15.7:0/1531611749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b1486c950 con 0x564b14949fb0 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60fde00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60f002cfd0 tx=0x7f60f002c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a9e5ecdfe22e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.386+0000 7f60e3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0042040 con 0x564b14949fb0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f60f0004ec0 con 0x564b14949fb0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60f002ca80 con 0x564b14949fb0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60f00050c0 con 0x564b14949fb0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f6107988740 10 client.16224 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b1483a820 con 0x564b14949fb0 2024-04-20T21:39:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.387+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f002ca80 con 0x564b14949fb0 2024-04-20T21:39:51.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f60f0003cc0 con 0x564b14949fb0 2024-04-20T21:39:51.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 10 client.16224 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 10 client.16224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 20 client.16224 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b14963330 0x564b14965720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 4 client.16224 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b148bca70 con 0x564b14963330 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.388+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b14963330 0x564b14965720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.389+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b14963330 0x564b14965720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29e4ae27a1393c30 server_cookie=3785bb797b9aed in_seq=0 out_seq=0 2024-04-20T21:39:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.389+0000 7f60e3400640 10 client.16224 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f60e3400640 1 -- 172.21.15.7:0/1531611749 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b148bca70 con 0x564b14963330 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f60e3400640 10 client.16224 handle_command_reply: tid=0 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 client.16224 shutdown 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 msgr2=0x564b14950550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f60f002cfd0 tx=0x7f60f002c330 comp rx=0 tx=0).stop 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 shutdown_connections 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b14963330 0x564b14965720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14950a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b149534b0 0x564b14950fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f60e3400640 0 client.16224 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 --2- 172.21.15.7:0/1531611749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b14949fb0 0x564b14950550 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 >> 172.21.15.7:0/1531611749 conn(0x564b149446e0 msgr2=0x564b14946430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 shutdown_connections 2024-04-20T21:39:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 1 -- 172.21.15.7:0/1531611749 wait complete. 2024-04-20T21:39:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.390+0000 7f6107988740 20 client.16224 trim_cache size 0 max 16384 2024-04-20T21:39:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.405+0000 7f6107988740 1 Processor -- start 2024-04-20T21:39:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.405+0000 7f6107988740 1 -- start start 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b14949fb0 0x564b14952c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b14968740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61001b2670 con 0x564b149534b0 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61001aff30 con 0x564b14949fb0 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f6107988740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61001adda0 con 0x564b1494a8d0 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60fe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b14968740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:35308/0 (socket says 172.21.15.7:35308) 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 -- 172.21.15.7:0/1223276947 learned_addr learned my addr 172.21.15.7:0/1223276947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60fde00640 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b14949fb0 0x564b14952c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 -- 172.21.15.7:0/1223276947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b14949fb0 msgr2=0x564b14952c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b14949fb0 0x564b14952c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 -- 172.21.15.7:0/1223276947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 msgr2=0x564b14968740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b14968740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.406+0000 7f60f7e00640 1 -- 172.21.15.7:0/1223276947 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b1494a700 con 0x564b1494a8d0 2024-04-20T21:39:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.407+0000 7f60f7e00640 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e8098d60 tx=0x7f60e80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c315d1098e49a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.407+0000 7f60ee600640 1 -- 172.21.15.7:0/1223276947 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80a4f50 con 0x564b1494a8d0 2024-04-20T21:39:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.407+0000 7f60ee600640 1 -- 172.21.15.7:0/1223276947 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e809a690 con 0x564b1494a8d0 2024-04-20T21:39:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.407+0000 7f60ee600640 1 -- 172.21.15.7:0/1223276947 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e809b7e0 con 0x564b1494a8d0 2024-04-20T21:39:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- 172.21.15.7:0/1223276947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 msgr2=0x564b149435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f60e8098d60 tx=0x7f60e80057f0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- 172.21.15.7:0/1223276947 shutdown_connections 2024-04-20T21:39:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b14968740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b1494a8d0 0x564b149435c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 --2- 172.21.15.7:0/1223276947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b14949fb0 0x564b14952c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- 172.21.15.7:0/1223276947 >> 172.21.15.7:0/1223276947 conn(0x564b149446e0 msgr2=0x564b14942eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- 172.21.15.7:0/1223276947 shutdown_connections 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- 172.21.15.7:0/1223276947 wait complete. 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 -- start start 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.408+0000 7f6107988740 1 Processor -- start 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14966e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b149c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61001ae3f0 con 0x564b149534b0 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61001af860 con 0x564b1494a8d0 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61001aeac0 con 0x564b14949fb0 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f6107988740 10 client.0 _ll_register_callbacks cb 0x7f60f0004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f60fde00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f60fde00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:35316/0 (socket says 172.21.15.7:35316) 2024-04-20T21:39:51.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f60fde00640 1 -- 172.21.15.7:0/4066614567 learned_addr learned my addr 172.21.15.7:0/4066614567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f60fe800640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b149c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.409+0000 7f60f7e00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14966e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 -- 172.21.15.7:0/4066614567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 msgr2=0x564b14966e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14966e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 -- 172.21.15.7:0/4066614567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 msgr2=0x564b149c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b149c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 -- 172.21.15.7:0/4066614567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b145c13f0 con 0x564b14949fb0 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60fde00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009900 tx=0x7f60f00349b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=913e283e4a91fe18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60e3400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0042040 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e809a690 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f60f00046c0 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f60f002c940 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.410+0000 7f6107988740 10 client.26395 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b1468b7d0 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e809a690 con 0x564b14949fb0 2024-04-20T21:39:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f60f00035f0 con 0x564b14949fb0 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 10 client.26395 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 10 client.26395 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 20 client.26395 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b149ca0e0 0x564b149cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 4 client.26395 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.411+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b148b68e0 con 0x564b149ca0e0 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.412+0000 7f60f7e00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b149ca0e0 0x564b149cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.412+0000 7f60f7e00640 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b149ca0e0 0x564b149cc500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e41fe48043294922 server_cookie=261cc9e73fa43c2a in_seq=0 out_seq=0 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.412+0000 7f60e3400640 10 client.26395 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f60e3400640 1 -- 172.21.15.7:0/4066614567 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b148b68e0 con 0x564b149ca0e0 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f60e3400640 10 client.26395 handle_command_reply: tid=0 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 client.26395 shutdown 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 msgr2=0x564b14966950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009900 tx=0x7f60f00349b0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 shutdown_connections 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564b149ca0e0 0x564b149cc500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b1494a8d0 0x564b14966e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f60e3400640 0 client.26395 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b149534b0 0x564b149c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 --2- 172.21.15.7:0/4066614567 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564b14949fb0 0x564b14966950 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 >> 172.21.15.7:0/4066614567 conn(0x564b149446e0 msgr2=0x564b1494e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 shutdown_connections 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 1 -- 172.21.15.7:0/4066614567 wait complete. 2024-04-20T21:39:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.413+0000 7f6107988740 20 client.26395 trim_cache size 0 max 16384 2024-04-20T21:39:51.421 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 msgr2=0x7f60e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 0x7f60e40a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005f40 tx=0x7f60f80096a0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 msgr2=0x7f61001a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d770 tx=0x7f60e800dc30 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 shutdown_connections 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f610019a8e0 0x7f610019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6100109470 0x7f61001a0c20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f60e40a3d20 0x7f60e40a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 --2- 172.21.15.7:0/3790960860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61001043a0 0x7f61001a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 >> 172.21.15.7:0/3790960860 conn(0x7f61000ffc20 msgr2=0x7f6100100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 shutdown_connections 2024-04-20T21:39:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.414+0000 7f6104c00640 1 -- 172.21.15.7:0/3790960860 wait complete. 2024-04-20T21:39:51.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:39:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.636+0000 7f2b09c00640 1 -- 172.21.15.7:0/1669855922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b041043b0 msgr2=0x7f2b04104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.636+0000 7f2b09c00640 1 --2- 172.21.15.7:0/1669855922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b041043b0 0x7f2b04104790 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009ca0 tx=0x7f2af402c100 comp rx=0 tx=0).stop 2024-04-20T21:39:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 -- 172.21.15.7:0/1669855922 shutdown_connections 2024-04-20T21:39:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 --2- 172.21.15.7:0/1669855922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b04109480 0x7f2b04111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 --2- 172.21.15.7:0/1669855922 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b04104d60 0x7f2b04108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 --2- 172.21.15.7:0/1669855922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b041043b0 0x7f2b04104790 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 -- 172.21.15.7:0/1669855922 >> 172.21.15.7:0/1669855922 conn(0x7f2b040ffbf0 msgr2=0x7f2b04102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 -- 172.21.15.7:0/1669855922 shutdown_connections 2024-04-20T21:39:51.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.637+0000 7f2b09c00640 1 -- 172.21.15.7:0/1669855922 wait complete. 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 Processor -- start 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 -- start start 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 0x7f2b041a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 0x7f2b0419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04114510 con 0x7f2b041043b0 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04114390 con 0x7f2b04109480 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.638+0000 7f2b09c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2b04114690 con 0x7f2b0419a870 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 0x7f2b041a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b03400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 0x7f2b0419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56598/0 (socket says 172.21.15.7:56598) 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 -- 172.21.15.7:0/3115748882 learned_addr learned my addr 172.21.15.7:0/3115748882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 0x7f2b041a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:33958/0 (socket says 172.21.15.7:33958) 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 -- 172.21.15.7:0/3115748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 msgr2=0x7f2b0419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 0x7f2b0419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 -- 172.21.15.7:0/3115748882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 msgr2=0x7f2b041a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 0x7f2b041a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b02a00640 1 -- 172.21.15.7:0/3115748882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0419b390 con 0x7f2b041043b0 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.639+0000 7f2b03400640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 0x7f2b0419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2b02a00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005e30 tx=0x7f2af4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2afb400640 1 -- 172.21.15.7:0/3115748882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4039040 con 0x7f2b041043b0 2024-04-20T21:39:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2afb400640 1 -- 172.21.15.7:0/3115748882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4044070 con 0x7f2b041043b0 2024-04-20T21:39:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2afb400640 1 -- 172.21.15.7:0/3115748882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40042a0 con 0x7f2b041043b0 2024-04-20T21:39:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0419b620 con 0x7f2b041043b0 2024-04-20T21:39:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.640+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b041a77c0 con 0x7f2b041043b0 2024-04-20T21:39:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.643+0000 7f2afb400640 1 -- 172.21.15.7:0/3115748882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2af4003970 con 0x7f2b041043b0 2024-04-20T21:39:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.643+0000 7f2b0c98c740 1 Processor -- start 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 -- start start 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 0x55cb553fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55404f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cb55361050 con 0x55cb554054b0 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb55307490 con 0x55cb553fbfb0 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb5535d010 con 0x55cb553fc8d0 2024-04-20T21:39:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b02a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55404f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.644+0000 7f2b02000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 0x55cb553fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b02000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56606/0 (socket says 172.21.15.7:56606) 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b02000640 1 -- 172.21.15.7:0/871847026 learned_addr learned my addr 172.21.15.7:0/871847026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b02a00640 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55404f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:35348/0 (socket says 172.21.15.7:35348) 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 -- 172.21.15.7:0/871847026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 msgr2=0x55cb55404f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55404f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 -- 172.21.15.7:0/871847026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 msgr2=0x55cb554078a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 -- 172.21.15.7:0/871847026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5530f070 con 0x55cb553fbfb0 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b02000640 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2afb400640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 0x7f2ae80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2b03400640 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 0x55cb553fc390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009f30 tx=0x7f2afc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518f201ed0d08038 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.645+0000 7f2afb400640 1 -- 172.21.15.7:0/3115748882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2af40f2ba0 con 0x7f2b041043b0 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.646+0000 7f2b02a00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 0x7f2ae80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.646+0000 7f2af3e00640 1 -- 172.21.15.7:0/871847026 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc004310 con 0x55cb553fbfb0 2024-04-20T21:39:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.646+0000 7f2af3e00640 1 -- 172.21.15.7:0/871847026 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc0044b0 con 0x55cb553fbfb0 2024-04-20T21:39:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.646+0000 7f2af3e00640 1 -- 172.21.15.7:0/871847026 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0035f0 con 0x55cb553fbfb0 2024-04-20T21:39:51.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- 172.21.15.7:0/871847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 msgr2=0x55cb553fc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 0x55cb553fc390 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2afc009f30 tx=0x7f2afc0118d0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b02a00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 0x7f2ae80a6260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55cb553fba80 tx=0x7f2af40f6650 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- 172.21.15.7:0/871847026 shutdown_connections 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb554054b0 0x55cb554078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55404f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/871847026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fbfb0 0x55cb553fc390 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- 172.21.15.7:0/871847026 >> 172.21.15.7:0/871847026 conn(0x55cb553f66e0 msgr2=0x55cb553f6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- 172.21.15.7:0/871847026 shutdown_connections 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- 172.21.15.7:0/871847026 wait complete. 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 -- start start 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.647+0000 7f2b0c98c740 1 Processor -- start 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55402ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb55402ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cb554082a0 con 0x55cb553fbfb0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cb5531a860 con 0x55cb554054b0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cb55408420 con 0x55cb553fc8d0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b0c98c740 10 client.0 _ll_register_callbacks cb 0x55cb552e8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b03400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b03400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56616/0 (socket says 172.21.15.7:56616) 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b03400640 1 -- 172.21.15.7:0/2593054083 learned_addr learned my addr 172.21.15.7:0/2593054083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.648+0000 7f2b02000640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55402ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb55402ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 -- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 msgr2=0x55cb55402ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55402ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 -- 172.21.15.7:0/2593054083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 msgr2=0x55cb55402ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb55402ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 -- 172.21.15.7:0/2593054083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5531e950 con 0x55cb553fbfb0 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2b03400640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2afc004bd0 tx=0x7f2afc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cee0e99ee5d040 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2af0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.649+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02c020 con 0x55cb553fbfb0 2024-04-20T21:39:51.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2afc003cb0 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2afc0044b0 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2afc004c70 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2b0c98c740 10 client.16242 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb552ec820 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2afc0044b0 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.650+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2afc022660 con 0x55cb553fbfb0 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 10 client.16242 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 10 client.16242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 20 client.16242 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb55415520 0x55cb55417910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 4 client.16242 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5536ea70 con 0x55cb55415520 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.651+0000 7f2b02000640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb55415520 0x55cb55417910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.652+0000 7f2b02000640 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb55415520 0x55cb55417910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea55a35a8f5c4a2a server_cookie=b7bc39b2edf21066 in_seq=0 out_seq=0 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.652+0000 7f2af0c00640 10 client.16242 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:51.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.652+0000 7f2af0c00640 1 -- 172.21.15.7:0/2593054083 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb5536ea70 con 0x55cb55415520 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.652+0000 7f2af0c00640 10 client.16242 handle_command_reply: tid=0 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 client.16242 shutdown 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 msgr2=0x55cb55402570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2afc004bd0 tx=0x7f2afc009310 comp rx=0 tx=0).stop 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 shutdown_connections 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb55415520 0x55cb55417910 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb55402ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fc8d0 0x55cb55402ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2593054083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55402570 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2af0c00640 0 client.16242 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 >> 172.21.15.7:0/2593054083 conn(0x55cb553f66e0 msgr2=0x55cb553f8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 shutdown_connections 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2593054083 wait complete. 2024-04-20T21:39:51.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.653+0000 7f2b0c98c740 20 client.16242 trim_cache size 0 max 16384 2024-04-20T21:39:51.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 Processor -- start 2024-04-20T21:39:51.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 -- start start 2024-04-20T21:39:51.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fbfb0 0x55cb55404ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb553f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041b2680 con 0x55cb553fc8d0 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041aff40 con 0x55cb554054b0 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.668+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041add90 con 0x55cb553fbfb0 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56622/0 (socket says 172.21.15.7:56622) 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 -- 172.21.15.7:0/2836245008 learned_addr learned my addr 172.21.15.7:0/2836245008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b03400640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fbfb0 0x55cb55404ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b03400640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb553f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 -- 172.21.15.7:0/2836245008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fbfb0 msgr2=0x55cb55404ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fbfb0 0x55cb55404ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 -- 172.21.15.7:0/2836245008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 msgr2=0x55cb553f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb553f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b02000640 1 -- 172.21.15.7:0/2836245008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb553fc700 con 0x55cb553fc8d0 2024-04-20T21:39:51.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.669+0000 7f2b03400640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb553f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.670+0000 7f2b02000640 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005e00 tx=0x7f2aec011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d7875ca49231ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.670+0000 7f2af3400640 1 -- 172.21.15.7:0/2836245008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec020190 con 0x55cb553fc8d0 2024-04-20T21:39:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.670+0000 7f2af3400640 1 -- 172.21.15.7:0/2836245008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aec024020 con 0x55cb553fc8d0 2024-04-20T21:39:51.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.670+0000 7f2af3400640 1 -- 172.21.15.7:0/2836245008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01ecc0 con 0x55cb553fc8d0 2024-04-20T21:39:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2836245008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 msgr2=0x55cb55411160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005e00 tx=0x7f2aec011980 comp rx=0 tx=0).stop 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2836245008 shutdown_connections 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb554054b0 0x55cb553f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fc8d0 0x55cb55411160 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/2836245008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb553fbfb0 0x55cb55404ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2836245008 >> 172.21.15.7:0/2836245008 conn(0x55cb553f66e0 msgr2=0x55cb553f4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2836245008 shutdown_connections 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- 172.21.15.7:0/2836245008 wait complete. 2024-04-20T21:39:51.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 -- start start 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.671+0000 7f2b0c98c740 1 Processor -- start 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fc8d0 0x55cb55409850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb554054b0 0x55cb55409d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041ae400 con 0x55cb553fbfb0 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041af870 con 0x55cb553fc8d0 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2b041aead0 con 0x55cb554054b0 2024-04-20T21:39:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b0c98c740 10 client.0 _ll_register_callbacks cb 0x7f2afc009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b03400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b03400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56626/0 (socket says 172.21.15.7:56626) 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b03400640 1 -- 172.21.15.7:0/1727988488 learned_addr learned my addr 172.21.15.7:0/1727988488 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b02000640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fc8d0 0x55cb55409850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.672+0000 7f2b03400640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb554054b0 0x55cb55409d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 -- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb554054b0 msgr2=0x55cb55409d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb554054b0 0x55cb55409d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 -- 172.21.15.7:0/1727988488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fc8d0 msgr2=0x55cb55409850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fc8d0 0x55cb55409850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 -- 172.21.15.7:0/1727988488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb550733f0 con 0x55cb553fbfb0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b03400640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2afc022fd0 tx=0x7f2afc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c98c05110bba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2af0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc02c040 con 0x55cb553fbfb0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2afc003cb0 con 0x55cb553fbfb0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.673+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec024020 con 0x55cb553fbfb0 2024-04-20T21:39:51.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2afc003eb0 con 0x55cb553fbfb0 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 10 client.16248 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb5513d7d0 con 0x55cb553fbfb0 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aec024020 con 0x55cb553fbfb0 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2afc0285b0 con 0x55cb553fbfb0 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 10 client.16248 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 10 client.16248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 20 client.16248 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb5547c080 0x55cb5547e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 4 client.16248 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.674+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb553688e0 con 0x55cb5547c080 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2b02000640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb5547c080 0x55cb5547e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2b02000640 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb5547c080 0x55cb5547e470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e279091166a40c5a server_cookie=a1f9b29b1895a524 in_seq=0 out_seq=0 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2af0c00640 10 client.16248 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:51.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2af0c00640 1 -- 172.21.15.7:0/1727988488 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb553688e0 con 0x55cb5547c080 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2af0c00640 10 client.16248 handle_command_reply: tid=0 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.675+0000 7f2b0c98c740 1 client.16248 shutdown 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 msgr2=0x55cb55409310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2afc022fd0 tx=0x7f2afc009310 comp rx=0 tx=0).stop 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 shutdown_connections 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x55cb5547c080 0x55cb5547e470 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cb553fc8d0 0x55cb55409850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cb554054b0 0x55cb55409d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2af0c00640 0 client.16248 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:39:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 --2- 172.21.15.7:0/1727988488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cb553fbfb0 0x55cb55409310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 >> 172.21.15.7:0/1727988488 conn(0x55cb553f66e0 msgr2=0x55cb553f5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 shutdown_connections 2024-04-20T21:39:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 1 -- 172.21.15.7:0/1727988488 wait complete. 2024-04-20T21:39:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.676+0000 7f2b0c98c740 20 client.16248 trim_cache size 0 max 16384 2024-04-20T21:39:51.685 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:51.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 msgr2=0x7f2ae80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 0x7f2ae80a6260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55cb553fba80 tx=0x7f2af40f6650 comp rx=0 tx=0).stop 2024-04-20T21:39:51.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 msgr2=0x7f2b041a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005e30 tx=0x7f2af4032f40 comp rx=0 tx=0).stop 2024-04-20T21:39:51.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 shutdown_connections 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2ae80a3da0 0x7f2ae80a6260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b04109480 0x7f2b041a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b0419a870 0x7f2b0419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 --2- 172.21.15.7:0/3115748882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b041043b0 0x7f2b041a06c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 >> 172.21.15.7:0/3115748882 conn(0x7f2b040ffbf0 msgr2=0x7f2b04101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 shutdown_connections 2024-04-20T21:39:51.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.677+0000 7f2b09c00640 1 -- 172.21.15.7:0/3115748882 wait complete. 2024-04-20T21:39:51.693 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:51 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/208953081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:51.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:39:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 -- 172.21.15.7:0/3613179162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 msgr2=0x7fbcd8115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 --2- 172.21.15.7:0/3613179162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd8115210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009cd0 tx=0x7fbcd002c0e0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 -- 172.21.15.7:0/3613179162 shutdown_connections 2024-04-20T21:39:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 --2- 172.21.15.7:0/3613179162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd8115210 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 --2- 172.21.15.7:0/3613179162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd81084a0 0x7fbcd8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 --2- 172.21.15.7:0/3613179162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd810e3e0 0x7fbcd810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 -- 172.21.15.7:0/3613179162 >> 172.21.15.7:0/3613179162 conn(0x7fbcd8077d70 msgr2=0x7fbcd8106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 -- 172.21.15.7:0/3613179162 shutdown_connections 2024-04-20T21:39:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.898+0000 7fbcd7200640 1 -- 172.21.15.7:0/3613179162 wait complete. 2024-04-20T21:39:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.899+0000 7fbcd7200640 1 Processor -- start 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.899+0000 7fbcd7200640 1 -- start start 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd81a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 0x7fbcd819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8118710 con 0x7fbcd8108e40 2024-04-20T21:39:51.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8118590 con 0x7fbcd81084a0 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd7200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8118890 con 0x7fbcd810e3e0 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd81a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 0x7fbcd819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.900+0000 7fbcd5e00640 1 -- 172.21.15.7:0/730529950 learned_addr learned my addr 172.21.15.7:0/730529950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 -- 172.21.15.7:0/730529950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 msgr2=0x7fbcd819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 0x7fbcd819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 -- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 msgr2=0x7fbcd81a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd81a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 -- 172.21.15.7:0/730529950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd819f130 con 0x7fbcd81084a0 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd6800640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 0x7fbcd819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800ea60 tx=0x7fbcc800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.901+0000 7fbccea00640 1 -- 172.21.15.7:0/730529950 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800cd20 con 0x7fbcd81084a0 2024-04-20T21:39:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.902+0000 7fbccea00640 1 -- 172.21.15.7:0/730529950 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc80035f0 con 0x7fbcd81084a0 2024-04-20T21:39:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.902+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd819f420 con 0x7fbcd81084a0 2024-04-20T21:39:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.902+0000 7fbccea00640 1 -- 172.21.15.7:0/730529950 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8010690 con 0x7fbcd81084a0 2024-04-20T21:39:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.902+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81ab6c0 con 0x7fbcd81084a0 2024-04-20T21:39:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.905+0000 7fbcdfde8740 1 Processor -- start 2024-04-20T21:39:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.905+0000 7fbcdfde8740 1 -- start start 2024-04-20T21:39:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135957130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135959a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e1358b3050 con 0x55e13594e170 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e135859490 con 0x55e135957670 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e1358af010 con 0x55e13594ea90 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbccea00640 1 -- 172.21.15.7:0/730529950 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbcc80037a0 con 0x7fbcd81084a0 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56654/0 (socket says 172.21.15.7:56654) 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcd6800640 1 -- 172.21.15.7:0/357070165 learned_addr learned my addr 172.21.15.7:0/357070165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.906+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135959a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 -- 172.21.15.7:0/357070165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 msgr2=0x55e135957130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135957130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135957130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 -- 172.21.15.7:0/357070165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 msgr2=0x55e135959a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135959a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 -- 172.21.15.7:0/357070165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e135861070 con 0x55e13594e170 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd5400640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135959a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135957130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.907+0000 7fbcd6800640 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcd002c6f0 tx=0x7fbcd0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24fb89996855116 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcc3400640 1 -- 172.21.15.7:0/357070165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0043070 con 0x55e13594e170 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcc3400640 1 -- 172.21.15.7:0/357070165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd0003e50 con 0x55e13594e170 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcc3400640 1 -- 172.21.15.7:0/357070165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0039040 con 0x55e13594e170 2024-04-20T21:39:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbccea00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 0x7fbcbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbccea00640 1 -- 172.21.15.7:0/730529950 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbcc80d0a00 con 0x7fbcd81084a0 2024-04-20T21:39:51.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 0x7fbcbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcdfde8740 1 -- 172.21.15.7:0/357070165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 msgr2=0x55e13594e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.908+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcd002c6f0 tx=0x7fbcd0005fd0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 -- 172.21.15.7:0/357070165 shutdown_connections 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135959a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135957130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/357070165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e13594e550 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 -- 172.21.15.7:0/357070165 >> 172.21.15.7:0/357070165 conn(0x55e1359488a0 msgr2=0x55e135948c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 -- 172.21.15.7:0/357070165 shutdown_connections 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 -- 172.21.15.7:0/357070165 wait complete. 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcd5e00640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 0x7fbcbc0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80ce250 tx=0x7fbcc800ef90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.909+0000 7fbcdfde8740 1 -- start start 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 Processor -- start 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135954bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135955130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e13595a400 con 0x55e13594e170 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e13586c860 con 0x55e135957670 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e13595a580 con 0x55e13594ea90 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.910+0000 7fbcdfde8740 10 client.0 _ll_register_callbacks cb 0x55e13583afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56666/0 (socket says 172.21.15.7:56666) 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 -- 172.21.15.7:0/2824914570 learned_addr learned my addr 172.21.15.7:0/2824914570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd5400640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135954bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135955130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 -- 172.21.15.7:0/2824914570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 msgr2=0x55e135954bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135954bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 -- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 msgr2=0x55e135955130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135955130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 -- 172.21.15.7:0/2824914570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e135870950 con 0x55e13594e170 2024-04-20T21:39:51.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd5400640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135954bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcd6800640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009bf0 tx=0x7fbcd0004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c942e8656a467cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.911+0000 7fbcb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0053060 con 0x55e13594e170 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbcd0032590 con 0x55e13594e170 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd0003e50 con 0x55e13594e170 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbcd0039040 con 0x55e13594e170 2024-04-20T21:39:51.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcdfde8740 10 client.16260 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e13583e820 con 0x55e13594e170 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.912+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd0003e50 con 0x55e13594e170 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fbcd003f440 con 0x55e13594e170 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 10 client.16260 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 20 client.16260 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e135967420 0x55e135969810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 4 client.16260 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1358c0a70 con 0x55e135967420 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.913+0000 7fbcd5400640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e135967420 0x55e135969810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.914+0000 7fbcd5400640 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e135967420 0x55e135969810 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ce5dfbf1bf21b90 server_cookie=dcdb58fc15f46186 in_seq=0 out_seq=0 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.914+0000 7fbcb7e00640 10 client.16260 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:51.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcb7e00640 1 -- 172.21.15.7:0/2824914570 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1358c0a70 con 0x55e135967420 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcb7e00640 10 client.16260 handle_command_reply: tid=0 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 client.16260 shutdown 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 msgr2=0x55e1359546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009bf0 tx=0x7fbcd0004280 comp rx=0 tx=0).stop 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 shutdown_connections 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e135955130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e135967420 0x55e135969810 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e135954bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/2824914570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e1359546b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 >> 172.21.15.7:0/2824914570 conn(0x55e1359488a0 msgr2=0x55e13594a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcb7e00640 0 client.16260 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 shutdown_connections 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 1 -- 172.21.15.7:0/2824914570 wait complete. 2024-04-20T21:39:51.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.915+0000 7fbcdfde8740 20 client.16260 trim_cache size 0 max 16384 2024-04-20T21:39:51.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 Processor -- start 2024-04-20T21:39:51.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 -- start start 2024-04-20T21:39:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e13596c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b6800 con 0x55e13594e170 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b40c0 con 0x55e135957670 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b1f10 con 0x55e13594ea90 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcd6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56668/0 (socket says 172.21.15.7:56668) 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.931+0000 7fbcd6800640 1 -- 172.21.15.7:0/3374722845 learned_addr learned my addr 172.21.15.7:0/3374722845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd5400640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e13596c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 -- 172.21.15.7:0/3374722845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 msgr2=0x55e13596c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e13596c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 -- 172.21.15.7:0/3374722845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 msgr2=0x55e13596cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 -- 172.21.15.7:0/3374722845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1355c53f0 con 0x55e13594e170 2024-04-20T21:39:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd5400640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e13596c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:39:51.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcd6800640 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcd002c620 tx=0x7fbcd002c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b699aa946ab814eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcc2a00640 1 -- 172.21.15.7:0/3374722845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd005b070 con 0x55e13594e170 2024-04-20T21:39:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcc2a00640 1 -- 172.21.15.7:0/3374722845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd00042f0 con 0x55e13594e170 2024-04-20T21:39:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.932+0000 7fbcc2a00640 1 -- 172.21.15.7:0/3374722845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0043040 con 0x55e13594e170 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 -- 172.21.15.7:0/3374722845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 msgr2=0x55e135956de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbcd002c620 tx=0x7fbcd002c0e0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 -- 172.21.15.7:0/3374722845 shutdown_connections 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594ea90 0x55e13596c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/3374722845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594e170 0x55e135956de0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 -- 172.21.15.7:0/3374722845 >> 172.21.15.7:0/3374722845 conn(0x55e1359488a0 msgr2=0x55e135963060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 -- 172.21.15.7:0/3374722845 shutdown_connections 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.933+0000 7fbcdfde8740 1 -- 172.21.15.7:0/3374722845 wait complete. 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.934+0000 7fbcdfde8740 1 -- start start 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.934+0000 7fbcdfde8740 1 Processor -- start 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.934+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594e170 0x55e13596cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.934+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.934+0000 7fbcdfde8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b2580 con 0x55e13594ea90 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b39f0 con 0x55e135957670 2024-04-20T21:39:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcdfde8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81b2c50 con 0x55e13594e170 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcdfde8740 10 client.0 _ll_register_callbacks cb 0x7fbcd0003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56680/0 (socket says 172.21.15.7:56680) 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 -- 172.21.15.7:0/78790469 learned_addr learned my addr 172.21.15.7:0/78790469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd6800640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd6800640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594e170 0x55e13596cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 -- 172.21.15.7:0/78790469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594e170 msgr2=0x55e13596cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594e170 0x55e13596cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 -- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 msgr2=0x55e13596a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.935+0000 7fbcd5400640 1 -- 172.21.15.7:0/78790469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13568f7d0 con 0x55e13594ea90 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcd5400640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4010540 tx=0x7fbcc4010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c74fe7518080d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcb7e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400fc80 con 0x55e13594ea90 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbcc400fe20 con 0x55e13594ea90 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcd00042f0 con 0x55e13594ea90 2024-04-20T21:39:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbcc402d020 con 0x55e13594ea90 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcdfde8740 10 client.16266 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1359cd510 con 0x55e13594ea90 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.936+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd00042f0 con 0x55e13594ea90 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fbcc40328a0 con 0x55e13594ea90 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 10 client.16266 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 20 client.16266 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e1359cdc40 0x55e1359d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 4 client.16266 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1358ba8e0 con 0x55e1359cdc40 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.937+0000 7fbcd6800640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e1359cdc40 0x55e1359d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcd6800640 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e1359cdc40 0x55e1359d0060 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6554f29bc2dd64d server_cookie=d8f7151fa73eea4b in_seq=0 out_seq=0 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcb7e00640 10 client.16266 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcb7e00640 1 -- 172.21.15.7:0/78790469 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1358ba8e0 con 0x55e1359cdc40 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcb7e00640 10 client.16266 handle_command_reply: tid=0 2024-04-20T21:39:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 client.16266 shutdown 2024-04-20T21:39:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 msgr2=0x55e135969ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4010540 tx=0x7fbcc4010d30 comp rx=0 tx=0).stop 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 shutdown_connections 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e135957670 0x55e13596a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e1359cdc40 0x55e1359d0060 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e13594ea90 0x55e135969ff0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 --2- 172.21.15.7:0/78790469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e13594e170 0x55e13596cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcb7e00640 0 client.16266 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 >> 172.21.15.7:0/78790469 conn(0x55e1359488a0 msgr2=0x55e135954670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 shutdown_connections 2024-04-20T21:39:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.938+0000 7fbcdfde8740 1 -- 172.21.15.7:0/78790469 wait complete. 2024-04-20T21:39:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcdfde8740 20 client.16266 trim_cache size 0 max 16384 2024-04-20T21:39:51.946 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 msgr2=0x7fbcbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 0x7fbcbc0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80ce250 tx=0x7fbcc800ef90 comp rx=0 tx=0).stop 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 msgr2=0x7fbcd81a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800ea60 tx=0x7fbcc800ee60 comp rx=0 tx=0).stop 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 shutdown_connections 2024-04-20T21:39:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbcd810e3e0 0x7fbcd819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcd8108e40 0x7fbcd81a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbcbc0a3d20 0x7fbcbc0a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 --2- 172.21.15.7:0/730529950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcd81084a0 0x7fbcd81a4800 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 >> 172.21.15.7:0/730529950 conn(0x7fbcd8077d70 msgr2=0x7fbcd8078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 shutdown_connections 2024-04-20T21:39:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:51.939+0000 7fbcd7200640 1 -- 172.21.15.7:0/730529950 wait complete. 2024-04-20T21:39:51.949 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:51 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/208953081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:51 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/208953081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:39:52.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:52 smithi007 ceph-mon[16867]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:52.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:52 smithi016 ceph-mon[21409]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:52.958 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:39:52.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:39:52.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:39:52.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:52 smithi186 ceph-mon[22157]: pgmap v199: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:39:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.187+0000 7f81e4400640 1 -- 172.21.15.7:0/111164549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc109470 msgr2=0x7f81dc111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 --2- 172.21.15.7:0/111164549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc109470 0x7f81dc111040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009ca0 tx=0x7f81d402c0f0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 -- 172.21.15.7:0/111164549 shutdown_connections 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 --2- 172.21.15.7:0/111164549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc109470 0x7f81dc111040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 --2- 172.21.15.7:0/111164549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc104d50 0x7f81dc108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 --2- 172.21.15.7:0/111164549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc1043a0 0x7f81dc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 -- 172.21.15.7:0/111164549 >> 172.21.15.7:0/111164549 conn(0x7f81dc0ffbe0 msgr2=0x7f81dc102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 -- 172.21.15.7:0/111164549 shutdown_connections 2024-04-20T21:39:53.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.188+0000 7f81e4400640 1 -- 172.21.15.7:0/111164549 wait complete. 2024-04-20T21:39:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 Processor -- start 2024-04-20T21:39:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 -- start start 2024-04-20T21:39:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 0x7f81dc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 0x7f81dc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc114510 con 0x7f81dc19a830 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc114390 con 0x7f81dc109470 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.189+0000 7f81e4400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc114690 con 0x7f81dc1043a0 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53088/0 (socket says 172.21.15.7:53088) 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 0x7f81dc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 -- 172.21.15.7:0/757202556 learned_addr learned my addr 172.21.15.7:0/757202556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e1000640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 0x7f81dc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 -- 172.21.15.7:0/757202556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 msgr2=0x7f81dc1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 0x7f81dc1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 -- 172.21.15.7:0/757202556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 msgr2=0x7f81dc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 0x7f81dc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e2400640 1 -- 172.21.15.7:0/757202556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc19b3c0 con 0x7f81dc19a830 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e1a00640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 0x7f81dc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.190+0000 7f81e1000640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 0x7f81dc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81e2400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f81d4037040 tx=0x7f81d4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81da000640 1 -- 172.21.15.7:0/757202556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4041020 con 0x7f81dc19a830 2024-04-20T21:39:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81da000640 1 -- 172.21.15.7:0/757202556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d4032e80 con 0x7f81dc19a830 2024-04-20T21:39:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc19b650 con 0x7f81dc19a830 2024-04-20T21:39:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81da000640 1 -- 172.21.15.7:0/757202556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4039040 con 0x7f81dc19a830 2024-04-20T21:39:53.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.191+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a7830 con 0x7f81dc19a830 2024-04-20T21:39:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 Processor -- start 2024-04-20T21:39:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81da000640 1 -- 172.21.15.7:0/757202556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f81d4044020 con 0x7f81dc19a830 2024-04-20T21:39:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 -- start start 2024-04-20T21:39:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 0x564cc653fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc6542800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cc649bd20 con 0x564cc6536f10 2024-04-20T21:39:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564cc64423e0 con 0x564cc6540410 2024-04-20T21:39:53.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.195+0000 7f81e6d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564cc6497ce0 con 0x564cc6537830 2024-04-20T21:39:53.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 0x564cc653fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 0x564cc653fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44858/0 (socket says 172.21.15.7:44858) 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1a00640 1 -- 172.21.15.7:0/3877723092 learned_addr learned my addr 172.21.15.7:0/3877723092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53102/0 (socket says 172.21.15.7:53102) 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc6542800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 -- 172.21.15.7:0/3877723092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 msgr2=0x564cc653fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 0x564cc653fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 -- 172.21.15.7:0/3877723092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 msgr2=0x564cc6542800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc6542800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.196+0000 7f81e1000640 1 -- 172.21.15.7:0/3877723092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc6461980 con 0x564cc6536f10 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81da000640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 0x7f81c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81e1000640 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009730 tx=0x7f81c80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116fcf1ac14d0258 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81da000640 1 -- 172.21.15.7:0/757202556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f81d40f2220 con 0x7f81dc19a830 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81e1a00640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 0x7f81c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81cea00640 1 -- 172.21.15.7:0/3877723092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8004030 con 0x564cc6536f10 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81cea00640 1 -- 172.21.15.7:0/3877723092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c800fd70 con 0x564cc6536f10 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.197+0000 7f81cea00640 1 -- 172.21.15.7:0/3877723092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8003a20 con 0x564cc6536f10 2024-04-20T21:39:53.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 -- 172.21.15.7:0/3877723092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 msgr2=0x564cc65372f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009730 tx=0x7f81c80096c0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e1a00640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 0x7f81c40a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x564cc65369e0 tx=0x7f81d0005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 -- 172.21.15.7:0/3877723092 shutdown_connections 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc6542800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6537830 0x564cc653fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3877723092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6536f10 0x564cc65372f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 -- 172.21.15.7:0/3877723092 >> 172.21.15.7:0/3877723092 conn(0x564cc6531640 msgr2=0x564cc6531a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 -- 172.21.15.7:0/3877723092 shutdown_connections 2024-04-20T21:39:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.198+0000 7f81e6d22740 1 -- 172.21.15.7:0/3877723092 wait complete. 2024-04-20T21:39:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 -- start start 2024-04-20T21:39:53.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 Processor -- start 2024-04-20T21:39:53.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.199+0000 7f81e6d22740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cc6543200 con 0x564cc6540410 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e6d22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564cc64557a0 con 0x564cc6537830 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e6d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564cc6543380 con 0x564cc6536f10 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e6d22740 10 client.0 _ll_register_callbacks cb 0x564cc6457b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34754/0 (socket says 172.21.15.7:34754) 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53114/0 (socket says 172.21.15.7:53114) 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 -- 172.21.15.7:0/481241032 learned_addr learned my addr 172.21.15.7:0/481241032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 -- 172.21.15.7:0/481241032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 msgr2=0x564cc653d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 -- 172.21.15.7:0/481241032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 msgr2=0x564cc653dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e1000640 1 -- 172.21.15.7:0/481241032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc62767e0 con 0x564cc6537830 2024-04-20T21:39:53.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.200+0000 7f81e2400640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81e1000640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c8003760 tx=0x7f81c8005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362d9cd400708c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81c3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8010840 con 0x564cc6537830 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f81c80109e0 con 0x564cc6537830 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c800fd70 con 0x564cc6537830 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81c8010be0 con 0x564cc6537830 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81e6d22740 10 client.26486 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:53.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.201+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cc62d6f70 con 0x564cc6537830 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c800fd70 con 0x564cc6537830 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f81c8028070 con 0x564cc6537830 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 10 client.26486 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 10 client.26486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 20 client.26486 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc6550620 0x564cc6552a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 4 client.26486 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cc64a9740 con 0x564cc6550620 2024-04-20T21:39:53.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.202+0000 7f81e1a00640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc6550620 0x564cc6552a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.203+0000 7f81e1a00640 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc6550620 0x564cc6552a10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1b46b4f514712c server_cookie=4d096e7398405d81 in_seq=0 out_seq=0 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.203+0000 7f81c3400640 10 client.26486 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.204+0000 7f81c3400640 1 -- 172.21.15.7:0/481241032 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cc64a9740 con 0x564cc6550620 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.204+0000 7f81c3400640 10 client.26486 handle_command_reply: tid=0 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.204+0000 7f81e6d22740 1 client.26486 shutdown 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.204+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 msgr2=0x564cc653d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.204+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f81c8003760 tx=0x7f81c8005750 comp rx=0 tx=0).stop 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 shutdown_connections 2024-04-20T21:39:53.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc653d980 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc6550620 0x564cc6552a10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 --2- 172.21.15.7:0/481241032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 >> 172.21.15.7:0/481241032 conn(0x564cc6531640 msgr2=0x564cc6533390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81c3400640 0 client.26486 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 shutdown_connections 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 1 -- 172.21.15.7:0/481241032 wait complete. 2024-04-20T21:39:53.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.205+0000 7f81e6d22740 20 client.26486 trim_cache size 0 max 16384 2024-04-20T21:39:53.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.224+0000 7f81e6d22740 1 Processor -- start 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.224+0000 7f81e6d22740 1 -- start start 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc654c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1b26a0 con 0x564cc6540410 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1aff60 con 0x564cc6537830 2024-04-20T21:39:53.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e6d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1addd0 con 0x564cc6536f10 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53130/0 (socket says 172.21.15.7:53130) 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e2400640 1 -- 172.21.15.7:0/3278390740 learned_addr learned my addr 172.21.15.7:0/3278390740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e1000640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc654c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.225+0000 7f81e1a00640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 -- 172.21.15.7:0/3278390740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 msgr2=0x564cc653fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 -- 172.21.15.7:0/3278390740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 msgr2=0x564cc654c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc654c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 -- 172.21.15.7:0/3278390740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc616e170 con 0x564cc6540410 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e1000640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc654c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:39:53.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81e2400640 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f81d40efb80 tx=0x7f81d40bde60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c324c4dd3961578a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81ce000640 1 -- 172.21.15.7:0/3278390740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40d7070 con 0x564cc6540410 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81ce000640 1 -- 172.21.15.7:0/3278390740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40f2040 con 0x564cc6540410 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.226+0000 7f81ce000640 1 -- 172.21.15.7:0/3278390740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40c6b00 con 0x564cc6540410 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 -- 172.21.15.7:0/3278390740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 msgr2=0x564cc653c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f81d40efb80 tx=0x7f81d40bde60 comp rx=0 tx=0).stop 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 -- 172.21.15.7:0/3278390740 shutdown_connections 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6537830 0x564cc654c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6540410 0x564cc653c5c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 --2- 172.21.15.7:0/3278390740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 -- 172.21.15.7:0/3278390740 >> 172.21.15.7:0/3278390740 conn(0x564cc6531640 msgr2=0x564cc652fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 -- 172.21.15.7:0/3278390740 shutdown_connections 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.227+0000 7f81e6d22740 1 -- 172.21.15.7:0/3278390740 wait complete. 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 -- start start 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 Processor -- start 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6537830 0x564cc653a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc653ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1ae420 con 0x564cc6537830 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1af890 con 0x564cc6540410 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc1aeaf0 con 0x564cc6536f10 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.228+0000 7f81e6d22740 10 client.0 _ll_register_callbacks cb 0x7f81d0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6537830 0x564cc653a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e1a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44878/0 (socket says 172.21.15.7:44878) 2024-04-20T21:39:53.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e1a00640 1 -- 172.21.15.7:0/2913044654 learned_addr learned my addr 172.21.15.7:0/2913044654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc653ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 -- 172.21.15.7:0/2913044654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 msgr2=0x564cc653a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 -- 172.21.15.7:0/2913044654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6537830 msgr2=0x564cc653a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6537830 0x564cc653a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 -- 172.21.15.7:0/2913044654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc65b5c10 con 0x564cc6540410 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81e2400640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc653ace0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009a60 tx=0x7f81d40c6d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468c2e662044ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.229+0000 7f81c3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40d7040 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d40f2040 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f81d40bc620 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81d40dd2b0 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 10 client.26492 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564cc65b6f30 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81d40f2040 con 0x564cc6540410 2024-04-20T21:39:53.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f81d403c030 con 0x564cc6540410 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 10 client.26492 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 20 client.26492 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc65b7550 0x564cc65b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 4 client.26492 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.230+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564cc64a35b0 con 0x564cc65b7550 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.231+0000 7f81e1a00640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc65b7550 0x564cc65b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.231+0000 7f81e1a00640 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc65b7550 0x564cc65b9940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1653c358512d19a3 server_cookie=a369b1f0529df42f in_seq=0 out_seq=0 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.231+0000 7f81c3400640 10 client.26492 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.232+0000 7f81c3400640 1 -- 172.21.15.7:0/2913044654 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564cc64a35b0 con 0x564cc65b7550 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.232+0000 7f81c3400640 10 client.26492 handle_command_reply: tid=0 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.232+0000 7f81e6d22740 1 client.26492 shutdown 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 msgr2=0x564cc653ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc653ace0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009a60 tx=0x7f81d40c6d30 comp rx=0 tx=0).stop 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 shutdown_connections 2024-04-20T21:39:53.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564cc6540410 0x564cc653ace0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x564cc65b7550 0x564cc65b9940 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cc6537830 0x564cc653a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 --2- 172.21.15.7:0/2913044654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564cc6536f10 0x564cc653a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 >> 172.21.15.7:0/2913044654 conn(0x564cc6531640 msgr2=0x564cc653dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81c3400640 0 client.26492 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 shutdown_connections 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 1 -- 172.21.15.7:0/2913044654 wait complete. 2024-04-20T21:39:53.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.233+0000 7f81e6d22740 20 client.26492 trim_cache size 0 max 16384 2024-04-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "85494cb4-65f5-4f6e-a270-c3dc3e59664f", 2024-04-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:39:53.242 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 msgr2=0x7f81c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 0x7f81c40a6260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x564cc65369e0 tx=0x7f81d0005f90 comp rx=0 tx=0).stop 2024-04-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 msgr2=0x7f81dc19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f81d4037040 tx=0x7f81d4032c90 comp rx=0 tx=0).stop 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 shutdown_connections 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f81c40a3da0 0x7f81c40a6260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc109470 0x7f81dc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81dc19a830 0x7f81dc19ace0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 --2- 172.21.15.7:0/757202556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f81dc1043a0 0x7f81dc1a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 >> 172.21.15.7:0/757202556 conn(0x7f81dc0ffbe0 msgr2=0x7f81dc101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 shutdown_connections 2024-04-20T21:39:53.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.234+0000 7f81e4400640 1 -- 172.21.15.7:0/757202556 wait complete. 2024-04-20T21:39:53.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:39:53.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:39:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 -- 172.21.15.7:0/3073885578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281041f0 msgr2=0x7f4c281045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 --2- 172.21.15.7:0/3073885578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281041f0 0x7f4c281045d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4c18009900 tx=0x7f4c1802bc40 comp rx=0 tx=0).stop 2024-04-20T21:39:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 -- 172.21.15.7:0/3073885578 shutdown_connections 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 --2- 172.21.15.7:0/3073885578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c28104ba0 0x7f4c28108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 --2- 172.21.15.7:0/3073885578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c281092c0 0x7f4c28110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 --2- 172.21.15.7:0/3073885578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281041f0 0x7f4c281045d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 -- 172.21.15.7:0/3073885578 >> 172.21.15.7:0/3073885578 conn(0x7f4c280ffa50 msgr2=0x7f4c28101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 -- 172.21.15.7:0/3073885578 shutdown_connections 2024-04-20T21:39:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.462+0000 7f4c27e00640 1 -- 172.21.15.7:0/3073885578 wait complete. 2024-04-20T21:39:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.463+0000 7f4c27e00640 1 Processor -- start 2024-04-20T21:39:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 -- start start 2024-04-20T21:39:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281092c0 0x7f4c281a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c2819a6c0 0x7f4c2819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281143a0 con 0x7f4c2819a6c0 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28114220 con 0x7f4c281041f0 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c28114520 con 0x7f4c281092c0 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c2819a6c0 0x7f4c2819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c26a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:34792/0 (socket says 172.21.15.7:34792) 2024-04-20T21:39:53.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c26a00640 1 -- 172.21.15.7:0/2145986261 learned_addr learned my addr 172.21.15.7:0/2145986261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.464+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281092c0 0x7f4c281a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 -- 172.21.15.7:0/2145986261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281092c0 msgr2=0x7f4c281a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281092c0 0x7f4c281a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 -- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c2819a6c0 msgr2=0x7f4c2819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c2819a6c0 0x7f4c2819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 -- 172.21.15.7:0/2145986261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2819b2c0 con 0x7f4c281041f0 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c26a00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005ab0 tx=0x7f4c18004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.465+0000 7f4c2ca00640 1 -- 172.21.15.7:0/2145986261 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18041070 con 0x7f4c281041f0 2024-04-20T21:39:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.466+0000 7f4c2ca00640 1 -- 172.21.15.7:0/2145986261 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1802cca0 con 0x7f4c281041f0 2024-04-20T21:39:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.466+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2819b550 con 0x7f4c281041f0 2024-04-20T21:39:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.466+0000 7f4c2ca00640 1 -- 172.21.15.7:0/2145986261 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003e50 con 0x7f4c281041f0 2024-04-20T21:39:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.466+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c281a7640 con 0x7f4c281041f0 2024-04-20T21:39:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.469+0000 7f4c2ca00640 1 -- 172.21.15.7:0/2145986261 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4c18044070 con 0x7f4c281041f0 2024-04-20T21:39:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.469+0000 7f4c30b86740 1 Processor -- start 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.469+0000 7f4c30b86740 1 -- start start 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1ca0 0x561f53eea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53eecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f53e47150 con 0x561f53eea880 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f53ded370 con 0x561f53ee1ca0 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c30b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561f53e43110 con 0x561f53ee1380 2024-04-20T21:39:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c26000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53eecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44900/0 (socket says 172.21.15.7:44900) 2024-04-20T21:39:53.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 -- 172.21.15.7:0/224664574 learned_addr learned my addr 172.21.15.7:0/224664574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c26a00640 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1ca0 0x561f53eea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 -- 172.21.15.7:0/224664574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1ca0 msgr2=0x561f53eea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1ca0 0x561f53eea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 -- 172.21.15.7:0/224664574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 msgr2=0x561f53eecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53eecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.470+0000 7f4c27400640 1 -- 172.21.15.7:0/224664574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f53c237d0 con 0x561f53ee1380 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c27400640 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007ec0 tx=0x7f4c2000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e65cffba2dcd626 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c2ca00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 0x7f4c0c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c13e00640 1 -- 172.21.15.7:0/224664574 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20010840 con 0x561f53ee1380 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c13e00640 1 -- 172.21.15.7:0/224664574 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c200183f0 con 0x561f53ee1380 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c13e00640 1 -- 172.21.15.7:0/224664574 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20017600 con 0x561f53ee1380 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c26a00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 0x7f4c0c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.471+0000 7f4c2ca00640 1 -- 172.21.15.7:0/2145986261 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c180f20e0 con 0x7f4c281041f0 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 -- 172.21.15.7:0/224664574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 msgr2=0x561f53ee1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c26a00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 0x7f4c0c0a6380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x561f53ee0e50 tx=0x7f4c180f5b90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4c20007ec0 tx=0x7f4c2000f790 comp rx=0 tx=0).stop 2024-04-20T21:39:53.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 -- 172.21.15.7:0/224664574 shutdown_connections 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1ca0 0x561f53eea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53eecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 --2- 172.21.15.7:0/224664574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1380 0x561f53ee1760 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 -- 172.21.15.7:0/224664574 >> 172.21.15.7:0/224664574 conn(0x561f53edbab0 msgr2=0x561f53edbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 -- 172.21.15.7:0/224664574 shutdown_connections 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.472+0000 7f4c30b86740 1 -- 172.21.15.7:0/224664574 wait complete. 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.473+0000 7f4c30b86740 1 -- start start 2024-04-20T21:39:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.473+0000 7f4c30b86740 1 Processor -- start 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1380 0x561f53ee7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53ee8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f53eed670 con 0x561f53eea880 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561f53e00740 con 0x561f53ee1380 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x561f53eed7f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c30b86740 10 client.0 _ll_register_callbacks cb 0x561f53dfbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44916/0 (socket says 172.21.15.7:44916) 2024-04-20T21:39:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 -- 172.21.15.7:0/621584748 learned_addr learned my addr 172.21.15.7:0/621584748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c27400640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1380 0x561f53ee7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c27400640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53ee8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 -- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1380 msgr2=0x561f53ee7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1380 0x561f53ee7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 -- 172.21.15.7:0/621584748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 msgr2=0x561f53ee8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53ee8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.474+0000 7f4c26000640 1 -- 172.21.15.7:0/621584748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f53b593f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c26000640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c14012700 tx=0x7f4c14012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa97b92e13a539f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c10c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400f920 con 0x561f53ee1ca0 2024-04-20T21:39:53.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c200183f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c1400fac0 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c1400fcc0 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c30b86740 10 client.26455 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.475+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f53b190f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c200183f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f4c1401c420 con 0x561f53ee1ca0 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 10 client.26455 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 10 client.26455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 20 client.26455 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53efad20 0x561f53efd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 4 client.26455 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.476+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f53e54b70 con 0x561f53efad20 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.477+0000 7f4c27400640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53efad20 0x561f53efd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.477+0000 7f4c27400640 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53efad20 0x561f53efd140 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c2f44d9b6023b server_cookie=b90d8b0dec15270e in_seq=0 out_seq=0 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.477+0000 7f4c10c00640 10 client.26455 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c10c00640 1 -- 172.21.15.7:0/621584748 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f53e54b70 con 0x561f53efad20 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c10c00640 10 client.26455 handle_command_reply: tid=0 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 client.26455 shutdown 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 msgr2=0x561f53ee7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c14012700 tx=0x7f4c14012bc0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 shutdown_connections 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53efad20 0x561f53efd140 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53eea880 0x561f53ee8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee7e50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 --2- 172.21.15.7:0/621584748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53ee1380 0x561f53ee7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c10c00640 0 client.26455 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 >> 172.21.15.7:0/621584748 conn(0x561f53edbab0 msgr2=0x561f53edd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 shutdown_connections 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.478+0000 7f4c30b86740 1 -- 172.21.15.7:0/621584748 wait complete. 2024-04-20T21:39:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.479+0000 7f4c30b86740 20 client.26455 trim_cache size 0 max 16384 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 Processor -- start 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 -- start start 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53eea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281b2470 con 0x561f53ee1380 2024-04-20T21:39:53.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281afd30 con 0x561f53eea880 2024-04-20T21:39:53.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.499+0000 7f4c30b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281adba0 con 0x561f53ee1ca0 2024-04-20T21:39:53.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44932/0 (socket says 172.21.15.7:44932) 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 -- 172.21.15.7:0/2160437116 learned_addr learned my addr 172.21.15.7:0/2160437116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c27400640 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53eea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 -- 172.21.15.7:0/2160437116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 msgr2=0x561f53ee2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 -- 172.21.15.7:0/2160437116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 msgr2=0x561f53eea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53eea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 -- 172.21.15.7:0/2160437116 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f53ef98b0 con 0x561f53ee1ca0 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.500+0000 7f4c26000640 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c1401ca50 tx=0x7f4c1400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447e9c7516aa65ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.501+0000 7f4c13400640 1 -- 172.21.15.7:0/2160437116 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14029070 con 0x561f53ee1ca0 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.501+0000 7f4c13400640 1 -- 172.21.15.7:0/2160437116 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1400f920 con 0x561f53ee1ca0 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.501+0000 7f4c13400640 1 -- 172.21.15.7:0/2160437116 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1402eba0 con 0x561f53ee1ca0 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- 172.21.15.7:0/2160437116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 msgr2=0x561f53ee6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c1401ca50 tx=0x7f4c1400fe60 comp rx=0 tx=0).stop 2024-04-20T21:39:53.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- 172.21.15.7:0/2160437116 shutdown_connections 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee6850 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 --2- 172.21.15.7:0/2160437116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53eea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- 172.21.15.7:0/2160437116 >> 172.21.15.7:0/2160437116 conn(0x561f53edbab0 msgr2=0x561f53edd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- 172.21.15.7:0/2160437116 shutdown_connections 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- 172.21.15.7:0/2160437116 wait complete. 2024-04-20T21:39:53.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 -- start start 2024-04-20T21:39:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.502+0000 7f4c30b86740 1 Processor -- start 2024-04-20T21:39:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53ee48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281ae1f0 con 0x561f53ee1380 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281af660 con 0x561f53eea880 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4c281ae8c0 con 0x561f53ee1ca0 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c30b86740 10 client.0 _ll_register_callbacks cb 0x7f4c14009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53ee48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c26000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:44948/0 (socket says 172.21.15.7:44948) 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c26000640 1 -- 172.21.15.7:0/1205243315 learned_addr learned my addr 172.21.15.7:0/1205243315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:39:53.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.503+0000 7f4c27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53ee48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53182/0 (socket says 172.21.15.7:53182) 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c27400640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 -- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 msgr2=0x561f53ee5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 -- 172.21.15.7:0/1205243315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 msgr2=0x561f53ee48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53ee48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 -- 172.21.15.7:0/1205243315 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f53f60500 con 0x561f53ee1ca0 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c26000640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c14012700 tx=0x7f4c1401c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88120e0a9269c5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:39:53.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c10c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.504+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14029070 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c140052c0 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c1400f920 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c1403c050 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c30b86740 10 client.26461 fetch_fsmap learned FSMap version 27 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f53f617f0 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1400f920 con 0x561f53ee1ca0 2024-04-20T21:39:53.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.505+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f4c1402e690 con 0x561f53ee1ca0 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 10 client.26461 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 20 client.26461 populate_metadata read hostname 'smithi007' 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53f61ea0 0x561f53f64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 4 client.26461 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f53e4e9e0 con 0x561f53f61ea0 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c27400640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53f61ea0 0x561f53f64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c27400640 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53f61ea0 0x561f53f64290 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5ed6417110e4f1 server_cookie=2b9cadd0483c5e28 in_seq=0 out_seq=0 2024-04-20T21:39:53.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.506+0000 7f4c10c00640 10 client.26461 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c10c00640 1 -- 172.21.15.7:0/1205243315 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561f53e4e9e0 con 0x561f53f61ea0 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c10c00640 10 client.26461 handle_command_reply: tid=0 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 client.26461 shutdown 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 msgr2=0x561f53ee4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c14012700 tx=0x7f4c1401c6f0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 shutdown_connections 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561f53eea880 0x561f53ee5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x561f53f61ea0 0x561f53f64290 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561f53ee1ca0 0x561f53ee4e20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 --2- 172.21.15.7:0/1205243315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f53ee1380 0x561f53ee48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c10c00640 0 client.26461 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 >> 172.21.15.7:0/1205243315 conn(0x561f53edbab0 msgr2=0x561f53ee78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 shutdown_connections 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 1 -- 172.21.15.7:0/1205243315 wait complete. 2024-04-20T21:39:53.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.507+0000 7f4c30b86740 20 client.26461 trim_cache size 0 max 16384 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "85494cb4-65f5-4f6e-a270-c3dc3e59664f": { 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "85494cb4-65f5-4f6e-a270-c3dc3e59664f", 2024-04-20T21:39:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:39:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:39:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:39:53.516 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 msgr2=0x7f4c0c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 0x7f4c0c0a6380 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x561f53ee0e50 tx=0x7f4c180f5b90 comp rx=0 tx=0).stop 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 msgr2=0x7f4c281a05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4c18005ab0 tx=0x7f4c18004360 comp rx=0 tx=0).stop 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 shutdown_connections 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c2819a6c0 0x7f4c2819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4c281092c0 0x7f4c281a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f4c0c0a3ec0 0x7f4c0c0a6380 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 --2- 172.21.15.7:0/2145986261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c281041f0 0x7f4c281a05b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:39:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 >> 172.21.15.7:0/2145986261 conn(0x7f4c280ffa50 msgr2=0x7f4c28100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:39:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 shutdown_connections 2024-04-20T21:39:53.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:39:53.508+0000 7f4c27e00640 1 -- 172.21.15.7:0/2145986261 wait complete. 2024-04-20T21:39:53.526 INFO:tasks.cephfs.filesystem:scrub status for tag:85494cb4-65f5-4f6e-a270-c3dc3e59664f - {'path': '/', 'tag': '85494cb4-65f5-4f6e-a270-c3dc3e59664f', 'options': 'recursive,force'} 2024-04-20T21:39:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:53.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:53 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:53.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:53.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:53 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:53.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:53.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:53 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:54 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:39:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:54 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:54 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:39:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:54 smithi007 ceph-mon[16867]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:54 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:54 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:39:54.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:54 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:54 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:39:54.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:54 smithi016 ceph-mon[21409]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:54.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:54 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:54 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:39:54.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:54 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:54.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:54 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:39:54.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:54 smithi186 ceph-mon[22157]: pgmap v200: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:39:54.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:54 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:39:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:56 smithi007 ceph-mon[16867]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-20T21:39:56.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:56 smithi016 ceph-mon[21409]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-20T21:39:56.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:56 smithi186 ceph-mon[22157]: pgmap v201: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-20T21:39:58.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:39:58 smithi007 ceph-mon[16867]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-04-20T21:39:58.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:39:58 smithi016 ceph-mon[21409]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-04-20T21:39:58.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:39:58 smithi186 ceph-mon[22157]: pgmap v202: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 5 op/s 2024-04-20T21:40:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:00 smithi007 ceph-mon[16867]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-20T21:40:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:00 smithi007 ceph-mon[16867]: overall HEALTH_OK 2024-04-20T21:40:00.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:00 smithi016 ceph-mon[21409]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-20T21:40:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:00 smithi016 ceph-mon[21409]: overall HEALTH_OK 2024-04-20T21:40:00.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:00 smithi186 ceph-mon[22157]: pgmap v203: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-20T21:40:00.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:00 smithi186 ceph-mon[22157]: overall HEALTH_OK 2024-04-20T21:40:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:02 smithi007 ceph-mon[16867]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-20T21:40:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:02 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:02 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:02 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:02.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:02 smithi016 ceph-mon[21409]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-20T21:40:02.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:02 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:02.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:02 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:02.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:02 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:02.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:02 smithi186 ceph-mon[22157]: pgmap v204: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-20T21:40:02.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:02 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:02.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:02 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:02.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:02 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:04 smithi007 ceph-mon[16867]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-04-20T21:40:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:04 smithi016 ceph-mon[21409]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-04-20T21:40:04.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:04 smithi186 ceph-mon[22157]: pgmap v205: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-04-20T21:40:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:06 smithi007 ceph-mon[16867]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-20T21:40:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:06 smithi016 ceph-mon[21409]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-20T21:40:06.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:06 smithi186 ceph-mon[22157]: pgmap v206: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-20T21:40:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:07 smithi016 ceph-mon[21409]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:40:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:07 smithi186 ceph-mon[22157]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:40:08.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:07 smithi007 ceph-mon[16867]: osdmap e90: 12 total, 12 up, 12 in 2024-04-20T21:40:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:08 smithi016 ceph-mon[21409]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:08.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:08 smithi186 ceph-mon[22157]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:09.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:08 smithi007 ceph-mon[16867]: pgmap v208: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:10.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:10 smithi016 ceph-mon[21409]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:10.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:10 smithi186 ceph-mon[22157]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:11.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:10 smithi007 ceph-mon[16867]: pgmap v209: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:12.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:12 smithi016 ceph-mon[21409]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:12 smithi186 ceph-mon[22157]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:13.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:12 smithi007 ceph-mon[16867]: pgmap v210: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:40:14.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:14 smithi016 ceph-mon[21409]: pgmap v211: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:14 smithi016 ceph-mon[21409]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:40:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:14 smithi186 ceph-mon[22157]: pgmap v211: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:14 smithi186 ceph-mon[22157]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:40:15.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:14 smithi007 ceph-mon[16867]: pgmap v211: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:40:15.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:14 smithi007 ceph-mon[16867]: osdmap e91: 12 total, 12 up, 12 in 2024-04-20T21:40:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:16 smithi016 ceph-mon[21409]: pgmap v213: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 351 B/s wr, 7 op/s 2024-04-20T21:40:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:16 smithi186 ceph-mon[22157]: pgmap v213: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 351 B/s wr, 7 op/s 2024-04-20T21:40:17.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:16 smithi007 ceph-mon[16867]: pgmap v213: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 351 B/s wr, 7 op/s 2024-04-20T21:40:17.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:17 smithi016 ceph-mon[21409]: pgmap v214: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:40:17.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:17 smithi186 ceph-mon[22157]: pgmap v214: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:40:18.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:17 smithi007 ceph-mon[16867]: pgmap v214: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:40:20.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:20 smithi016 ceph-mon[21409]: pgmap v215: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:20.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:20 smithi186 ceph-mon[22157]: pgmap v215: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:20.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:20 smithi007 ceph-mon[16867]: pgmap v215: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:22.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:22 smithi016 ceph-mon[21409]: pgmap v216: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:22.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:22 smithi186 ceph-mon[22157]: pgmap v216: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:22.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:22 smithi007 ceph-mon[16867]: pgmap v216: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:40:23.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:23.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:23.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.756+0000 7f5040800640 1 -- 172.21.15.7:0/772795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5038109470 msgr2=0x7f5038111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.756+0000 7f5040800640 1 --2- 172.21.15.7:0/772795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5038109470 0x7f5038111040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5034009900 tx=0x7f503402bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 -- 172.21.15.7:0/772795952 shutdown_connections 2024-04-20T21:40:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 --2- 172.21.15.7:0/772795952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038104d50 0x7f5038108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 --2- 172.21.15.7:0/772795952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5038109470 0x7f5038111040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 --2- 172.21.15.7:0/772795952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f50381043a0 0x7f5038104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 -- 172.21.15.7:0/772795952 >> 172.21.15.7:0/772795952 conn(0x7f50380ffbe0 msgr2=0x7f5038102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 -- 172.21.15.7:0/772795952 shutdown_connections 2024-04-20T21:40:23.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.757+0000 7f5040800640 1 -- 172.21.15.7:0/772795952 wait complete. 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 Processor -- start 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 -- start start 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038109470 0x7f50381a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f503819a8f0 0x7f503819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5038114530 con 0x7f50381043a0 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50381143b0 con 0x7f5038109470 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.758+0000 7f5040800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f50381146b0 con 0x7f503819a8f0 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39574/0 (socket says 172.21.15.7:39574) 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 -- 172.21.15.7:0/3845642405 learned_addr learned my addr 172.21.15.7:0/3845642405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503d400640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038109470 0x7f50381a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503e800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f503819a8f0 0x7f503819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 -- 172.21.15.7:0/3845642405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f503819a8f0 msgr2=0x7f503819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f503819a8f0 0x7f503819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 -- 172.21.15.7:0/3845642405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038109470 msgr2=0x7f50381a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038109470 0x7f50381a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.759+0000 7f503de00640 1 -- 172.21.15.7:0/3845642405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503819b410 con 0x7f50381043a0 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f503de00640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f503000eb90 tx=0x7f503000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f502ea00640 1 -- 172.21.15.7:0/3845642405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030004d20 con 0x7f50381043a0 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f502ea00640 1 -- 172.21.15.7:0/3845642405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503000ee90 con 0x7f50381043a0 2024-04-20T21:40:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f502ea00640 1 -- 172.21.15.7:0/3845642405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030010830 con 0x7f50381043a0 2024-04-20T21:40:23.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503819b700 con 0x7f50381043a0 2024-04-20T21:40:23.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.760+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a7900 con 0x7f50381043a0 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.763+0000 7f5043469740 1 Processor -- start 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.763+0000 7f5043469740 1 -- start start 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f502ea00640 1 -- 172.21.15.7:0/3845642405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5030005020 con 0x7f50381043a0 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c9d150 con 0x5639a9d37ac0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c43370 con 0x5639a9d371a0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f5043469740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c99110 con 0x5639a9d406a0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50802/0 (socket says 172.21.15.7:50802) 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503d400640 1 -- 172.21.15.7:0/2826657363 learned_addr learned my addr 172.21.15.7:0/2826657363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.764+0000 7f503d400640 1 -- 172.21.15.7:0/2826657363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 msgr2=0x5639a9d37580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f503d400640 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d37580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f503d400640 1 -- 172.21.15.7:0/2826657363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 msgr2=0x5639a9d40160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f503d400640 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d40160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f503d400640 1 -- 172.21.15.7:0/2826657363 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a9a797d0 con 0x5639a9d406a0 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f503d400640 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5028009ba0 tx=0x7f5028011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1187b758040d5b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f5023000640 1 -- 172.21.15.7:0/2826657363 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028004230 con 0x5639a9d406a0 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f5023000640 1 -- 172.21.15.7:0/2826657363 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028009e10 con 0x5639a9d406a0 2024-04-20T21:40:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.765+0000 7f5023000640 1 -- 172.21.15.7:0/2826657363 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028004da0 con 0x5639a9d406a0 2024-04-20T21:40:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f502ea00640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 0x7f50240a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f502ea00640 1 -- 172.21.15.7:0/3845642405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f50300d0da0 con 0x7f50381043a0 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 -- 172.21.15.7:0/2826657363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 msgr2=0x5639a9d42a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5028009ba0 tx=0x7f5028011810 comp rx=0 tx=0).stop 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f503e800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 0x7f50240a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 -- 172.21.15.7:0/2826657363 shutdown_connections 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d42a90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 --2- 172.21.15.7:0/2826657363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 -- 172.21.15.7:0/2826657363 >> 172.21.15.7:0/2826657363 conn(0x5639a9d318d0 msgr2=0x5639a9d31cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 -- 172.21.15.7:0/2826657363 shutdown_connections 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.766+0000 7f5043469740 1 -- 172.21.15.7:0/2826657363 wait complete. 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 -- start start 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f503e800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 0x7f50240a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5034005ab0 tx=0x7f503402e040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 Processor -- start 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d3dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5639a9d43490 con 0x5639a9d37ac0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5639a9c56740 con 0x5639a9d371a0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.767+0000 7f5043469740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5639a9d43610 con 0x5639a9d406a0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f5043469740 10 client.0 _ll_register_callbacks cb 0x5639a9c51b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38308/0 (socket says 172.21.15.7:38308) 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 -- 172.21.15.7:0/1432170880 learned_addr learned my addr 172.21.15.7:0/1432170880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503d400640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d3dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503e800640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 -- 172.21.15.7:0/1432170880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 msgr2=0x5639a9d3e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 -- 172.21.15.7:0/1432170880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 msgr2=0x5639a9d3dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d3dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 -- 172.21.15.7:0/1432170880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a99af3f0 con 0x5639a9d371a0 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f503de00640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f503000cba0 tx=0x7f5030004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27436b14d9491cff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.768+0000 7f5017400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503009c8a0 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f50300993d0 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5028009e10 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f503009c6c0 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5043469740 10 client.26519 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639a996f0f0 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028009e10 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.769+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f50300a5540 con 0x5639a9d371a0 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 10 client.26519 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 10 client.26519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 20 client.26519 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9d509f0 0x5639a9d52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 4 client.26519 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a9caab70 con 0x5639a9d509f0 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.770+0000 7f503d400640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9d509f0 0x5639a9d52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.771+0000 7f503d400640 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9d509f0 0x5639a9d52e60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42123a906d3de7fe server_cookie=fdf2d9632d7497e3 in_seq=0 out_seq=0 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.771+0000 7f5017400640 10 client.26519 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.771+0000 7f5017400640 1 -- 172.21.15.7:0/1432170880 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639a9caab70 con 0x5639a9d509f0 2024-04-20T21:40:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.771+0000 7f5017400640 10 client.26519 handle_command_reply: tid=0 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.771+0000 7f5043469740 1 client.26519 shutdown 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 msgr2=0x5639a9d3d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f503000cba0 tx=0x7f5030004ec0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 shutdown_connections 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9d509f0 0x5639a9d52e60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d3dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 --2- 172.21.15.7:0/1432170880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d3d660 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 >> 172.21.15.7:0/1432170880 conn(0x5639a9d318d0 msgr2=0x5639a9d33620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5017400640 0 client.26519 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 shutdown_connections 2024-04-20T21:40:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 1 -- 172.21.15.7:0/1432170880 wait complete. 2024-04-20T21:40:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.772+0000 7f5043469740 20 client.26519 trim_cache size 0 max 16384 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.791+0000 7f5043469740 1 Processor -- start 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 -- start start 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d37ac0 0x5639a9d4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 0x5639a9d33410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b26e0 con 0x5639a9d371a0 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50381affa0 con 0x5639a9d406a0 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f5043469740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ade10 con 0x5639a9d37ac0 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 0x5639a9d33410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 0x5639a9d33410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38320/0 (socket says 172.21.15.7:38320) 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503e800640 1 -- 172.21.15.7:0/4106166852 learned_addr learned my addr 172.21.15.7:0/4106166852 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39596/0 (socket says 172.21.15.7:39596) 2024-04-20T21:40:23.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.792+0000 7f503d400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d37ac0 0x5639a9d4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 -- 172.21.15.7:0/4106166852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d37ac0 msgr2=0x5639a9d4c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d37ac0 0x5639a9d4c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 -- 172.21.15.7:0/4106166852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 msgr2=0x5639a9d33410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 0x5639a9d33410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 -- 172.21.15.7:0/4106166852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a9d31400 con 0x5639a9d371a0 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f503de00640 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f50300a54a0 tx=0x7f503009b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf5d17655420350 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f5022600640 1 -- 172.21.15.7:0/4106166852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503009c8a0 con 0x5639a9d371a0 2024-04-20T21:40:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f5022600640 1 -- 172.21.15.7:0/4106166852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503009d460 con 0x5639a9d371a0 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.793+0000 7f5022600640 1 -- 172.21.15.7:0/4106166852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50300993d0 con 0x5639a9d371a0 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- 172.21.15.7:0/4106166852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 msgr2=0x5639a9d3fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f50300a54a0 tx=0x7f503009b270 comp rx=0 tx=0).stop 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- 172.21.15.7:0/4106166852 shutdown_connections 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d406a0 0x5639a9d33410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d37ac0 0x5639a9d4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 --2- 172.21.15.7:0/4106166852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d371a0 0x5639a9d3fd90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- 172.21.15.7:0/4106166852 >> 172.21.15.7:0/4106166852 conn(0x5639a9d318d0 msgr2=0x5639a9d32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- 172.21.15.7:0/4106166852 shutdown_connections 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- 172.21.15.7:0/4106166852 wait complete. 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.794+0000 7f5043469740 1 -- start start 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 Processor -- start 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d48af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ae460 con 0x5639a9d37ac0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50381af8d0 con 0x5639a9d371a0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aeb30 con 0x5639a9d406a0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f5043469740 10 client.0 _ll_register_callbacks cb 0x7f5030004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f503d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f503d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39610/0 (socket says 172.21.15.7:39610) 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.795+0000 7f503d400640 1 -- 172.21.15.7:0/2511705533 learned_addr learned my addr 172.21.15.7:0/2511705533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503e800640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503de00640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d48af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 -- 172.21.15.7:0/2511705533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 msgr2=0x5639a9d3a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 -- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 msgr2=0x5639a9d48af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d48af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 -- 172.21.15.7:0/2511705533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a9d3aad0 con 0x5639a9d37ac0 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f503d400640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50280099b0 tx=0x7f50280049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c1b012fb1f6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f5017400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.796+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502801e040 con 0x5639a9d37ac0 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f5028004050 con 0x5639a9d37ac0 2024-04-20T21:40:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503009d460 con 0x5639a9d37ac0 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5028004270 con 0x5639a9d37ac0 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 10 client.16317 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5639a9db6e70 con 0x5639a9d37ac0 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503009d460 con 0x5639a9d37ac0 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f5028036020 con 0x5639a9d37ac0 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 10 client.16317 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 20 client.16317 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9db73b0 0x5639a9db97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 4 client.16317 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.797+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639a9ca49e0 con 0x5639a9db73b0 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f503de00640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9db73b0 0x5639a9db97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f503de00640 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9db73b0 0x5639a9db97d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8d6d240bbd627b server_cookie=36de612f280421d0 in_seq=0 out_seq=0 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f5017400640 10 client.16317 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f5017400640 1 -- 172.21.15.7:0/2511705533 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5639a9ca49e0 con 0x5639a9db73b0 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f5017400640 10 client.16317 handle_command_reply: tid=0 2024-04-20T21:40:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.798+0000 7f5043469740 1 client.16317 shutdown 2024-04-20T21:40:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 msgr2=0x5639a9d39ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f50280099b0 tx=0x7f50280049a0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 shutdown_connections 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5639a9db73b0 0x5639a9db97d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5639a9d406a0 0x5639a9d3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5639a9d37ac0 0x5639a9d39ef0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 --2- 172.21.15.7:0/2511705533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5639a9d371a0 0x5639a9d48af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 >> 172.21.15.7:0/2511705533 conn(0x5639a9d318d0 msgr2=0x5639a9d33590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5017400640 0 client.16317 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 shutdown_connections 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 1 -- 172.21.15.7:0/2511705533 wait complete. 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.799+0000 7f5043469740 20 client.16317 trim_cache size 0 max 16384 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:40:23.806 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 msgr2=0x7f50240a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 0x7f50240a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5034005ab0 tx=0x7f503402e040 comp rx=0 tx=0).stop 2024-04-20T21:40:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 msgr2=0x7f50381a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f503000eb90 tx=0x7f503000c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 shutdown_connections 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f50240a3da0 0x7f50240a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5038109470 0x7f50381a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f503819a8f0 0x7f503819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 --2- 172.21.15.7:0/3845642405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50381043a0 0x7f50381a06f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 >> 172.21.15.7:0/3845642405 conn(0x7f50380ffbe0 msgr2=0x7f5038101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 shutdown_connections 2024-04-20T21:40:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:23.800+0000 7f5040800640 1 -- 172.21.15.7:0/3845642405 wait complete. 2024-04-20T21:40:23.816 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:40:23.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 -- 172.21.15.7:0/3413536772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc10e500 msgr2=0x7f7dfc10e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 --2- 172.21.15.7:0/3413536772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc10e500 0x7f7dfc10e8e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009900 tx=0x7f7df002bcc0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 -- 172.21.15.7:0/3413536772 shutdown_connections 2024-04-20T21:40:24.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 --2- 172.21.15.7:0/3413536772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 --2- 172.21.15.7:0/3413536772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc1084a0 0x7f7dfc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 --2- 172.21.15.7:0/3413536772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc10e500 0x7f7dfc10e8e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 -- 172.21.15.7:0/3413536772 >> 172.21.15.7:0/3413536772 conn(0x7f7dfc077d30 msgr2=0x7f7dfc106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 -- 172.21.15.7:0/3413536772 shutdown_connections 2024-04-20T21:40:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.024+0000 7f7e02e00640 1 -- 172.21.15.7:0/3413536772 wait complete. 2024-04-20T21:40:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.025+0000 7f7e02e00640 1 Processor -- start 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 -- start start 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc1084a0 0x7f7dfc1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 0x7f7dfc19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc118840 con 0x7f7dfc108e40 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1186c0 con 0x7f7dfc10e500 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e02e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1189c0 con 0x7f7dfc1084a0 2024-04-20T21:40:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.026+0000 7f7e01000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 0x7f7dfc19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e01000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39638/0 (socket says 172.21.15.7:39638) 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e01000640 1 -- 172.21.15.7:0/2067713533 learned_addr learned my addr 172.21.15.7:0/2067713533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e01a00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc1084a0 0x7f7dfc1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 -- 172.21.15.7:0/2067713533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc1084a0 msgr2=0x7f7dfc1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc1084a0 0x7f7dfc1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 -- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 msgr2=0x7f7dfc1a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 -- 172.21.15.7:0/2067713533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc19f3e0 con 0x7f7dfc10e500 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.027+0000 7f7e02400640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 0x7f7dfc19eb20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7df400ec50 tx=0x7f7df400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.028+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4003b70 con 0x7f7dfc10e500 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.028+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc19f6d0 con 0x7f7dfc10e500 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.028+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df4003d10 con 0x7f7dfc10e500 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.028+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4010890 con 0x7f7dfc10e500 2024-04-20T21:40:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.028+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1ab9e0 con 0x7f7dfc10e500 2024-04-20T21:40:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.029+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc109b90 con 0x7f7dfc10e500 2024-04-20T21:40:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.031+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7df4003fc0 con 0x7f7dfc10e500 2024-04-20T21:40:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.032+0000 7f7dfa000640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 0x7f7de40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.033+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f7df40d0d50 con 0x7f7dfc10e500 2024-04-20T21:40:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.033+0000 7f7e01a00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 0x7f7de40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.034+0000 7f7e01a00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 0x7f7de40a61d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005ab0 tx=0x7f7df0005a20 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.037+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f7df409d3b0 con 0x7f7dfc10e500 2024-04-20T21:40:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.278+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dfc04fd60 con 0x7f7dfc10e500 2024-04-20T21:40:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.279+0000 7f7dfa000640 1 -- 172.21.15.7:0/2067713533 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f7df409ccb0 con 0x7f7dfc10e500 2024-04-20T21:40:24.278 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:40:24.279 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:40:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:40:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.282+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 msgr2=0x7f7de40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.282+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 0x7f7de40a61d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7df0005ab0 tx=0x7f7df0005a20 comp rx=0 tx=0).stop 2024-04-20T21:40:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.282+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 msgr2=0x7f7dfc19eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.282+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 0x7f7dfc19eb20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7df400ec50 tx=0x7f7df400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 shutdown_connections 2024-04-20T21:40:24.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7de40a3d10 0x7f7de40a61d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc10e500 0x7f7dfc19eb20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7dfc108e40 0x7f7dfc1a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 --2- 172.21.15.7:0/2067713533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7dfc1084a0 0x7f7dfc1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 >> 172.21.15.7:0/2067713533 conn(0x7f7dfc077d30 msgr2=0x7f7dfc078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 shutdown_connections 2024-04-20T21:40:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.283+0000 7f7e02e00640 1 -- 172.21.15.7:0/2067713533 wait complete. 2024-04-20T21:40:24.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:40:24.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 -- 172.21.15.7:0/78550499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 msgr2=0x7ff94c105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 --2- 172.21.15.7:0/78550499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c105da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009be0 tx=0x7ff93c02bf80 comp rx=0 tx=0).stop 2024-04-20T21:40:24.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 -- 172.21.15.7:0/78550499 shutdown_connections 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 --2- 172.21.15.7:0/78550499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c1062e0 0x7ff94c106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 --2- 172.21.15.7:0/78550499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c105da0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 --2- 172.21.15.7:0/78550499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c100fc0 0x7ff94c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 -- 172.21.15.7:0/78550499 >> 172.21.15.7:0/78550499 conn(0x7ff94c078010 msgr2=0x7ff94c078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 -- 172.21.15.7:0/78550499 shutdown_connections 2024-04-20T21:40:24.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.510+0000 7ff952400640 1 -- 172.21.15.7:0/78550499 wait complete. 2024-04-20T21:40:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.511+0000 7ff952400640 1 Processor -- start 2024-04-20T21:40:24.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 -- start start 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 0x7ff94c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c19f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c118940 con 0x7ff94c1062e0 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1187c0 con 0x7ff94c100fc0 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff952400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c118ac0 con 0x7ff94c1018e0 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.512+0000 7ff94be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39668/0 (socket says 172.21.15.7:39668) 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 0x7ff94c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 -- 172.21.15.7:0/3375508656 learned_addr learned my addr 172.21.15.7:0/3375508656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94aa00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c19f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 -- 172.21.15.7:0/3375508656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 msgr2=0x7ff94c19f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c19f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 -- 172.21.15.7:0/3375508656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 msgr2=0x7ff94c19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 0x7ff94c19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 -- 172.21.15.7:0/3375508656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c1a3e10 con 0x7ff94c1062e0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94b400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 0x7ff94c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.513+0000 7ff94be00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e970 tx=0x7ff94400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.514+0000 7ff943e00640 1 -- 172.21.15.7:0/3375508656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400ccc0 con 0x7ff94c1062e0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.514+0000 7ff943e00640 1 -- 172.21.15.7:0/3375508656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94400ce60 con 0x7ff94c1062e0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.514+0000 7ff943e00640 1 -- 172.21.15.7:0/3375508656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944010430 con 0x7ff94c1062e0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.514+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c1a4100 con 0x7ff94c1062e0 2024-04-20T21:40:24.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.514+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1ab990 con 0x7ff94c1062e0 2024-04-20T21:40:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.517+0000 7ff9551e5740 1 Processor -- start 2024-04-20T21:40:24.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.517+0000 7ff943e00640 1 -- 172.21.15.7:0/3375508656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff944005020 con 0x7ff94c1062e0 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 -- start start 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b3334f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b33354b0 0x5600b33378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5600b3291050 con 0x5600b332bfb0 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600b3237490 con 0x5600b33354b0 2024-04-20T21:40:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff9551e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5600b328d010 con 0x5600b332c8d0 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39676/0 (socket says 172.21.15.7:39676) 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff94aa00640 1 -- 172.21.15.7:0/2890972839 learned_addr learned my addr 172.21.15.7:0/2890972839 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.518+0000 7ff94b400640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b3334f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94aa00640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b33354b0 0x5600b33378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 -- 172.21.15.7:0/2890972839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b33354b0 msgr2=0x5600b33378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b33354b0 0x5600b33378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 -- 172.21.15.7:0/2890972839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 msgr2=0x5600b332c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 -- 172.21.15.7:0/2890972839 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b323f070 con 0x5600b332c8d0 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94aa00640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff943e00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 0x7ff9300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff943e00640 1 -- 172.21.15.7:0/3375508656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff9440d0a50 con 0x7ff94c1062e0 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94b400640 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b3334f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff934005fd0 tx=0x7ff9340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe114fd84ce7e30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff94aa00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 0x7ff9300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.519+0000 7ff940c00640 1 -- 172.21.15.7:0/2890972839 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934004020 con 0x5600b332c8d0 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff940c00640 1 -- 172.21.15.7:0/2890972839 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9340041c0 con 0x5600b332c8d0 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff940c00640 1 -- 172.21.15.7:0/2890972839 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93400f900 con 0x5600b332c8d0 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 -- 172.21.15.7:0/2890972839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 msgr2=0x5600b3334f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff94aa00640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 0x7ff9300a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5600b3327870 tx=0x7ff93c033440 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b3334f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff934005fd0 tx=0x7ff9340118d0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 -- 172.21.15.7:0/2890972839 shutdown_connections 2024-04-20T21:40:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b33354b0 0x5600b33378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b3334f70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2890972839 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332bfb0 0x5600b332c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 -- 172.21.15.7:0/2890972839 >> 172.21.15.7:0/2890972839 conn(0x5600b33266e0 msgr2=0x5600b3326ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.520+0000 7ff9551e5740 1 -- 172.21.15.7:0/2890972839 shutdown_connections 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.521+0000 7ff9551e5740 1 -- 172.21.15.7:0/2890972839 wait complete. 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.521+0000 7ff9551e5740 1 -- start start 2024-04-20T21:40:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.521+0000 7ff9551e5740 1 Processor -- start 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3332520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3332a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b3332fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5600b33382a0 con 0x5600b332c8d0 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600b324a860 con 0x5600b332bfb0 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5600b3338420 con 0x5600b33354b0 2024-04-20T21:40:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff9551e5740 10 client.0 _ll_register_callbacks cb 0x5600b3218fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3332a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3332a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39690/0 (socket says 172.21.15.7:39690) 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94aa00640 1 -- 172.21.15.7:0/2438693369 learned_addr learned my addr 172.21.15.7:0/2438693369 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b3332fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94b400640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3332520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 -- 172.21.15.7:0/2438693369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 msgr2=0x5600b3332520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3332520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 -- 172.21.15.7:0/2438693369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 msgr2=0x5600b3332a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3332a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.522+0000 7ff94be00640 1 -- 172.21.15.7:0/2438693369 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b324e950 con 0x5600b33354b0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff94be00640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b3332fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff9440ce130 tx=0x7ff94409c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0ac9effe769e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff939600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440a0d60 con 0x5600b33354b0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9340041c0 con 0x5600b33354b0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff944005310 con 0x5600b33354b0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff9440a5620 con 0x5600b33354b0 2024-04-20T21:40:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff9551e5740 10 client.26497 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600b321c820 con 0x5600b33354b0 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.523+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9440b8030 con 0x5600b33354b0 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ff94409dce0 con 0x5600b33354b0 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 10 client.26497 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 10 client.26497 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 20 client.26497 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b3345510 0x5600b3347920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 4 client.26497 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b329ea70 con 0x5600b3345510 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.524+0000 7ff94b400640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b3345510 0x5600b3347920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.525+0000 7ff94b400640 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b3345510 0x5600b3347920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f22becc77ae052b6 server_cookie=c39c0cd8e5b1d937 in_seq=0 out_seq=0 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.525+0000 7ff939600640 10 client.26497 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff939600640 1 -- 172.21.15.7:0/2438693369 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600b329ea70 con 0x5600b3345510 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff939600640 10 client.26497 handle_command_reply: tid=0 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 client.26497 shutdown 2024-04-20T21:40:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 msgr2=0x5600b3332fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b3332fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff9440ce130 tx=0x7ff94409c3d0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 shutdown_connections 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b3332fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3332a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b3345510 0x5600b3347920 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 --2- 172.21.15.7:0/2438693369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3332520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 >> 172.21.15.7:0/2438693369 conn(0x5600b33266e0 msgr2=0x5600b3328430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff939600640 0 client.26497 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 shutdown_connections 2024-04-20T21:40:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 1 -- 172.21.15.7:0/2438693369 wait complete. 2024-04-20T21:40:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.526+0000 7ff9551e5740 20 client.26497 trim_cache size 0 max 16384 2024-04-20T21:40:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.541+0000 7ff9551e5740 1 Processor -- start 2024-04-20T21:40:24.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.541+0000 7ff9551e5740 1 -- start start 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3334c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3341190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b6b50 con 0x5600b332c8d0 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b4410 con 0x5600b332bfb0 2024-04-20T21:40:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff9551e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b2280 con 0x5600b33354b0 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3341190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50874/0 (socket says 172.21.15.7:50874) 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 -- 172.21.15.7:0/514669260 learned_addr learned my addr 172.21.15.7:0/514669260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94b400640 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3334c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 -- 172.21.15.7:0/514669260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 msgr2=0x5600b3334c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3334c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 -- 172.21.15.7:0/514669260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 msgr2=0x5600b3341190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.542+0000 7ff94be00640 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3341190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff94be00640 1 -- 172.21.15.7:0/514669260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b332c700 con 0x5600b33354b0 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff94aa00640 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3341190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff94be00640 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff944009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0ef01bc72fcb8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff93be00640 1 -- 172.21.15.7:0/514669260 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94407f020 con 0x5600b33354b0 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff93be00640 1 -- 172.21.15.7:0/514669260 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9440b8070 con 0x5600b33354b0 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.543+0000 7ff93be00640 1 -- 172.21.15.7:0/514669260 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9440a0d60 con 0x5600b33354b0 2024-04-20T21:40:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- 172.21.15.7:0/514669260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 msgr2=0x5600b33255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff944009760 comp rx=0 tx=0).stop 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- 172.21.15.7:0/514669260 shutdown_connections 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b33354b0 0x5600b33255c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b332c8d0 0x5600b3341190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 --2- 172.21.15.7:0/514669260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3334c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- 172.21.15.7:0/514669260 >> 172.21.15.7:0/514669260 conn(0x5600b33266e0 msgr2=0x5600b3324eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- 172.21.15.7:0/514669260 shutdown_connections 2024-04-20T21:40:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- 172.21.15.7:0/514669260 wait complete. 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.544+0000 7ff9551e5740 1 -- start start 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 Processor -- start 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3339060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 0x5600b3339ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b28d0 con 0x5600b33354b0 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b3d40 con 0x5600b332bfb0 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1b2fa0 con 0x5600b332c8d0 2024-04-20T21:40:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff9551e5740 10 client.0 _ll_register_callbacks cb 0x7ff94409c7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff94b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3339060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff94be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 0x5600b3339ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.545+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3339060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38400/0 (socket says 172.21.15.7:38400) 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94b400640 1 -- 172.21.15.7:0/1534421706 learned_addr learned my addr 172.21.15.7:0/1534421706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50884/0 (socket says 172.21.15.7:50884) 2024-04-20T21:40:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 -- 172.21.15.7:0/1534421706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 msgr2=0x5600b3339060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3339060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 -- 172.21.15.7:0/1534421706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 msgr2=0x5600b3339ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 0x5600b3339ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 -- 172.21.15.7:0/1534421706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b2fa33f0 con 0x5600b332c8d0 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94be00640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 0x5600b3339ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff94aa00640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5600b3333af0 tx=0x7ff93c02bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a042bde27aed2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.546+0000 7ff939600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0047d0 con 0x5600b332c8d0 2024-04-20T21:40:24.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9440b8070 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff93c033520 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff93c033720 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff9551e5740 10 client.26503 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.547+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600b306d7d0 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9440b8070 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7ff93c043a10 con 0x5600b332c8d0 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 10 client.26503 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 10 client.26503 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:40:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 20 client.26503 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b33ac110 0x5600b33ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 4 client.26503 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600b32988e0 con 0x5600b33ac110 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.548+0000 7ff94b400640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b33ac110 0x5600b33ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff94b400640 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b33ac110 0x5600b33ae530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb113888e2ec4645 server_cookie=170d3ffd71939113 in_seq=0 out_seq=0 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff939600640 10 client.26503 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff939600640 1 -- 172.21.15.7:0/1534421706 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600b32988e0 con 0x5600b33ac110 2024-04-20T21:40:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff939600640 10 client.26503 handle_command_reply: tid=0 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 client.26503 shutdown 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 msgr2=0x5600b33395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5600b3333af0 tx=0x7ff93c02bf80 comp rx=0 tx=0).stop 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 shutdown_connections 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5600b33354b0 0x5600b3339ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5600b332c8d0 0x5600b33395a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5600b33ac110 0x5600b33ae530 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 --2- 172.21.15.7:0/1534421706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5600b332bfb0 0x5600b3339060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 >> 172.21.15.7:0/1534421706 conn(0x5600b33266e0 msgr2=0x5600b3325740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff939600640 0 client.26503 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 shutdown_connections 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.549+0000 7ff9551e5740 1 -- 172.21.15.7:0/1534421706 wait complete. 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff9551e5740 20 client.26503 trim_cache size 0 max 16384 2024-04-20T21:40:24.559 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 msgr2=0x7ff9300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 0x7ff9300a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5600b3327870 tx=0x7ff93c033440 comp rx=0 tx=0).stop 2024-04-20T21:40:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 msgr2=0x7ff94c1a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff94400e970 tx=0x7ff94400ee30 comp rx=0 tx=0).stop 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 shutdown_connections 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff94c1062e0 0x7ff94c1a36e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff94c1018e0 0x7ff94c19f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff9300a3da0 0x7ff9300a6260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 --2- 172.21.15.7:0/3375508656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c100fc0 0x7ff94c19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 >> 172.21.15.7:0/3375508656 conn(0x7ff94c078010 msgr2=0x7ff94c103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.550+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 shutdown_connections 2024-04-20T21:40:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.551+0000 7ff952400640 1 -- 172.21.15.7:0/3375508656 wait complete. 2024-04-20T21:40:24.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:24 smithi007 ceph-mon[16867]: pgmap v217: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:40:24.564 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:24 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2067713533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:24.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:40:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:24 smithi016 ceph-mon[21409]: pgmap v217: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:40:24.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:24 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2067713533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:24 smithi186 ceph-mon[22157]: pgmap v217: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:40:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:24 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2067713533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 -- 172.21.15.7:0/3465096502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0103860 msgr2=0x7f29f010bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 --2- 172.21.15.7:0/3465096502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0103860 0x7f29f010bd40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009900 tx=0x7f29dc02bcc0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 -- 172.21.15.7:0/3465096502 shutdown_connections 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 --2- 172.21.15.7:0/3465096502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0103860 0x7f29f010bd40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 --2- 172.21.15.7:0/3465096502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f010c360 0x7f29f010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 --2- 172.21.15.7:0/3465096502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0102f40 0x7f29f0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 -- 172.21.15.7:0/3465096502 >> 172.21.15.7:0/3465096502 conn(0x7f29f0077a10 msgr2=0x7f29f00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 -- 172.21.15.7:0/3465096502 shutdown_connections 2024-04-20T21:40:24.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.774+0000 7f29f7000640 1 -- 172.21.15.7:0/3465096502 wait complete. 2024-04-20T21:40:24.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.775+0000 7f29f7000640 1 Processor -- start 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 -- start start 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0103860 0x7f29f01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f010c360 0x7f29f019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01120f0 con 0x7f29f0102f40 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0111f70 con 0x7f29f010c360 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f7000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0112270 con 0x7f29f0103860 2024-04-20T21:40:24.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39730/0 (socket says 172.21.15.7:39730) 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f5c00640 1 -- 172.21.15.7:0/3213736703 learned_addr learned my addr 172.21.15.7:0/3213736703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f6600640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f010c360 0x7f29f019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.776+0000 7f29f5200640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0103860 0x7f29f01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 -- 172.21.15.7:0/3213736703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0103860 msgr2=0x7f29f01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0103860 0x7f29f01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 -- 172.21.15.7:0/3213736703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f010c360 msgr2=0x7f29f019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f010c360 0x7f29f019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 -- 172.21.15.7:0/3213736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f019b010 con 0x7f29f0102f40 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.777+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f29e000e9f0 tx=0x7f29e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.778+0000 7f29ee000640 1 -- 172.21.15.7:0/3213736703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e000cd80 con 0x7f29f0102f40 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.778+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f019b300 con 0x7f29f0102f40 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.778+0000 7f29ee000640 1 -- 172.21.15.7:0/3213736703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0010910 con 0x7f29f0102f40 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.778+0000 7f29ee000640 1 -- 172.21.15.7:0/3213736703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e00188f0 con 0x7f29f0102f40 2024-04-20T21:40:24.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.778+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a75c0 con 0x7f29f0102f40 2024-04-20T21:40:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.781+0000 7f29fb219740 1 Processor -- start 2024-04-20T21:40:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.781+0000 7f29ee000640 1 -- 172.21.15.7:0/3213736703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f29e0010ac0 con 0x7f29f0102f40 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.781+0000 7f29fb219740 1 -- start start 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf93130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573aaeef050 con 0x5573aaf93670 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5573aae95490 con 0x5573aaf8a170 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29fb219740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573aaeeb010 con 0x5573aaf8aa90 2024-04-20T21:40:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf93130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39738/0 (socket says 172.21.15.7:39738) 2024-04-20T21:40:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5200640 1 -- 172.21.15.7:0/2567759309 learned_addr learned my addr 172.21.15.7:0/2567759309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5200640 1 -- 172.21.15.7:0/2567759309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 msgr2=0x5573aaf93130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.782+0000 7f29f5200640 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf93130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f5200640 1 -- 172.21.15.7:0/2567759309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 msgr2=0x5573aaf8a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f5200640 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f5200640 1 -- 172.21.15.7:0/2567759309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573aae9d070 con 0x5573aaf93670 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f5c00640 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f6600640 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29f5200640 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02c200 tx=0x7f29dc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b393f482a249c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29e6a00640 1 -- 172.21.15.7:0/2567759309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc045070 con 0x5573aaf93670 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29e6a00640 1 -- 172.21.15.7:0/2567759309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29dc004040 con 0x5573aaf93670 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.783+0000 7f29e6a00640 1 -- 172.21.15.7:0/2567759309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc004c20 con 0x5573aaf93670 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29ee000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 0x7f29d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29ee000640 1 -- 172.21.15.7:0/3213736703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f29e00d0b20 con 0x7f29f0102f40 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 -- 172.21.15.7:0/2567759309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 msgr2=0x5573aaf95a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f29dc02c200 tx=0x7f29dc005dd0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29f6600640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 0x7f29d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 -- 172.21.15.7:0/2567759309 shutdown_connections 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf95a60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf93130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 --2- 172.21.15.7:0/2567759309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 -- 172.21.15.7:0/2567759309 >> 172.21.15.7:0/2567759309 conn(0x5573aaf848a0 msgr2=0x5573aaf84c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 -- 172.21.15.7:0/2567759309 shutdown_connections 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.784+0000 7f29fb219740 1 -- 172.21.15.7:0/2567759309 wait complete. 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.785+0000 7f29f6600640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 0x7f29d40a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009c70 tx=0x7f29e8009600 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.785+0000 7f29fb219740 1 -- start start 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.785+0000 7f29fb219740 1 Processor -- start 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 0x5573aaf91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5573aaf96400 con 0x5573aaf8a170 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5573aaea8860 con 0x5573aaf93670 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5573aaf96580 con 0x5573aaf8aa90 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29fb219740 10 client.0 _ll_register_callbacks cb 0x5573aae76fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:24.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 0x5573aaf91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39748/0 (socket says 172.21.15.7:39748) 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5c00640 1 -- 172.21.15.7:0/1946813611 learned_addr learned my addr 172.21.15.7:0/1946813611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 -- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 msgr2=0x5573aaf90c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf90c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 -- 172.21.15.7:0/1946813611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 msgr2=0x5573aaf906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f6600640 1 -- 172.21.15.7:0/1946813611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573aaeac950 con 0x5573aaf93670 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5200640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.786+0000 7f29f5c00640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29f6600640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 0x5573aaf91150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f29e8011d00 tx=0x7f29e8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=befe4710d3e10f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29db400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8028600 con 0x5573aaf93670 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29dc004040 con 0x5573aaf93670 2024-04-20T21:40:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f29e80287a0 con 0x5573aaf93670 2024-04-20T21:40:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f29e80289a0 con 0x5573aaf93670 2024-04-20T21:40:24.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29fb219740 10 client.26561 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.787+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573aae7a820 con 0x5573aaf93670 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29dc004040 con 0x5573aaf93670 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f29e802f020 con 0x5573aaf93670 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 10 client.26561 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 10 client.26561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 20 client.26561 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573aafa3640 0x5573aafa5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 4 client.26561 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573aaefca70 con 0x5573aafa3640 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.788+0000 7f29f5c00640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573aafa3640 0x5573aafa5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.789+0000 7f29f5c00640 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573aafa3640 0x5573aafa5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=729f03f701004837 server_cookie=9ab7f071d9384133 in_seq=0 out_seq=0 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.789+0000 7f29db400640 10 client.26561 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29db400640 1 -- 172.21.15.7:0/1946813611 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573aaefca70 con 0x5573aafa3640 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29db400640 10 client.26561 handle_command_reply: tid=0 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 client.26561 shutdown 2024-04-20T21:40:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 msgr2=0x5573aaf91150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 0x5573aaf91150 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f29e8011d00 tx=0x7f29e8004a40 comp rx=0 tx=0).stop 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 shutdown_connections 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573aafa3640 0x5573aafa5a90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf93670 0x5573aaf91150 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 --2- 172.21.15.7:0/1946813611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf8a170 0x5573aaf906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29db400640 0 client.26561 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 >> 172.21.15.7:0/1946813611 conn(0x5573aaf848a0 msgr2=0x5573aaf865f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 shutdown_connections 2024-04-20T21:40:24.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 1 -- 172.21.15.7:0/1946813611 wait complete. 2024-04-20T21:40:24.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.790+0000 7f29fb219740 20 client.26561 trim_cache size 0 max 16384 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 Processor -- start 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 -- start start 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf9f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b2570 con 0x5573aaf93670 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01afe30 con 0x5573aaf8a170 2024-04-20T21:40:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.805+0000 7f29fb219740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01adca0 con 0x5573aaf8aa90 2024-04-20T21:40:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39764/0 (socket says 172.21.15.7:39764) 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 -- 172.21.15.7:0/3181867555 learned_addr learned my addr 172.21.15.7:0/3181867555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f5200640 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf9f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 -- 172.21.15.7:0/3181867555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 msgr2=0x5573aaf9f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf9f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 -- 172.21.15.7:0/3181867555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 msgr2=0x5573aaf92e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf92e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 -- 172.21.15.7:0/3181867555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573aac013f0 con 0x5573aaf93670 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.806+0000 7f29f6600640 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f29e80096e0 tx=0x7f29e8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963bdd688f9cdf5a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29e6000640 1 -- 172.21.15.7:0/3181867555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8028b40 con 0x5573aaf93670 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29e6000640 1 -- 172.21.15.7:0/3181867555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8028ce0 con 0x5573aaf93670 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29e6000640 1 -- 172.21.15.7:0/3181867555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8003bd0 con 0x5573aaf93670 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 -- 172.21.15.7:0/3181867555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 msgr2=0x5573aaf837e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f29e80096e0 tx=0x7f29e8011f40 comp rx=0 tx=0).stop 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 -- 172.21.15.7:0/3181867555 shutdown_connections 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573aaf837e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf9f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 --2- 172.21.15.7:0/3181867555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf92e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.807+0000 7f29fb219740 1 -- 172.21.15.7:0/3181867555 >> 172.21.15.7:0/3181867555 conn(0x5573aaf848a0 msgr2=0x5573aaf830d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.808+0000 7f29fb219740 1 -- 172.21.15.7:0/3181867555 shutdown_connections 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.808+0000 7f29fb219740 1 -- 172.21.15.7:0/3181867555 wait complete. 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.808+0000 7f29fb219740 1 -- start start 2024-04-20T21:40:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.808+0000 7f29fb219740 1 Processor -- start 2024-04-20T21:40:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ae2f0 con 0x5573aaf93670 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01af760 con 0x5573aaf8a170 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01ae9c0 con 0x5573aaf8aa90 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29fb219740 10 client.0 _ll_register_callbacks cb 0x7f29e80119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39776/0 (socket says 172.21.15.7:39776) 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 -- 172.21.15.7:0/3840689363 learned_addr learned my addr 172.21.15.7:0/3840689363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f5200640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 -- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 msgr2=0x5573aaf8df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf8df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 -- 172.21.15.7:0/3840689363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 msgr2=0x5573aaf8d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.809+0000 7f29f6600640 1 -- 172.21.15.7:0/3840689363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573aaccb7d0 con 0x5573aaf93670 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29f6600640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f29e80042f0 tx=0x7f29e80114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c788beaea8b750 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29db400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80287e0 con 0x5573aaf93670 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f29e8003970 con 0x5573aaf93670 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e8028ce0 con 0x5573aaf93670 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f29e804a050 con 0x5573aaf93670 2024-04-20T21:40:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29fb219740 10 client.16359 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.810+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573ab00a0e0 con 0x5573aaf93670 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e8028ce0 con 0x5573aaf93670 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f29e8039030 con 0x5573aaf93670 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 20 client.16359 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573ab00a6f0 0x5573ab00cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 4 client.16359 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573aaef68e0 con 0x5573ab00a6f0 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.811+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573ab00a6f0 0x5573ab00cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29f5c00640 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573ab00a6f0 0x5573ab00cb10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdac9b9ba789a570 server_cookie=24603dd8c37852b1 in_seq=0 out_seq=0 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29db400640 10 client.16359 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29db400640 1 -- 172.21.15.7:0/3840689363 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573aaef68e0 con 0x5573ab00a6f0 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29db400640 10 client.16359 handle_command_reply: tid=0 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 client.16359 shutdown 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 msgr2=0x5573ab008bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f29e80042f0 tx=0x7f29e80114c0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 shutdown_connections 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5573ab00a6f0 0x5573ab00cb10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5573aaf93670 0x5573ab008bd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5573aaf8aa90 0x5573aaf8df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 --2- 172.21.15.7:0/3840689363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5573aaf8a170 0x5573aaf8d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 >> 172.21.15.7:0/3840689363 conn(0x5573aaf848a0 msgr2=0x5573aaf83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29db400640 0 client.16359 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 shutdown_connections 2024-04-20T21:40:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.812+0000 7f29fb219740 1 -- 172.21.15.7:0/3840689363 wait complete. 2024-04-20T21:40:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29fb219740 20 client.16359 trim_cache size 0 max 16384 2024-04-20T21:40:24.819 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 msgr2=0x7f29d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 0x7f29d40a6260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009c70 tx=0x7f29e8009600 comp rx=0 tx=0).stop 2024-04-20T21:40:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 msgr2=0x7f29f01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f29e000e9f0 tx=0x7f29e000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 shutdown_connections 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f29d40a3da0 0x7f29d40a6260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f010c360 0x7f29f019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29f0103860 0x7f29f01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 --2- 172.21.15.7:0/3213736703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29f0102f40 0x7f29f01a0640 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 >> 172.21.15.7:0/3213736703 conn(0x7f29f0077a10 msgr2=0x7f29f0100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 shutdown_connections 2024-04-20T21:40:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:24.813+0000 7f29f7000640 1 -- 172.21.15.7:0/3213736703 wait complete. 2024-04-20T21:40:24.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:40:25.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 -- 172.21.15.7:0/3969970391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8100b00 msgr2=0x7f62a8105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 --2- 172.21.15.7:0/3969970391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8100b00 0x7f62a8105110 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6294009900 tx=0x7f629402bc40 comp rx=0 tx=0).stop 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 -- 172.21.15.7:0/3969970391 shutdown_connections 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 --2- 172.21.15.7:0/3969970391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a810ae70 0x7f62a810d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 --2- 172.21.15.7:0/3969970391 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8100b00 0x7f62a8105110 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 --2- 172.21.15.7:0/3969970391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a8068580 0x7f62a8104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.030+0000 7f62a7e00640 1 -- 172.21.15.7:0/3969970391 >> 172.21.15.7:0/3969970391 conn(0x7f62a8077dd0 msgr2=0x7f62a80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.031+0000 7f62a7e00640 1 -- 172.21.15.7:0/3969970391 shutdown_connections 2024-04-20T21:40:25.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.031+0000 7f62a7e00640 1 -- 172.21.15.7:0/3969970391 wait complete. 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.032+0000 7f62a7e00640 1 Processor -- start 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.032+0000 7f62a7e00640 1 -- start start 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.032+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8068580 0x7f62a81a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.032+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.032+0000 7f62a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 0x7f62a8109b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8112170 con 0x7f62a8100b00 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8111ff0 con 0x7f62a810ae70 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a7e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81122f0 con 0x7f62a8068580 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 0x7f62a8109b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39802/0 (socket says 172.21.15.7:39802) 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 -- 172.21.15.7:0/658444008 learned_addr learned my addr 172.21.15.7:0/658444008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6a00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8068580 0x7f62a81a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 -- 172.21.15.7:0/658444008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8068580 msgr2=0x7f62a81a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8068580 0x7f62a81a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 -- 172.21.15.7:0/658444008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 msgr2=0x7f62a8109b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 0x7f62a8109b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a6000640 1 -- 172.21.15.7:0/658444008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a810a3b0 con 0x7f62a8100b00 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.033+0000 7f62a7400640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 0x7f62a8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.034+0000 7f62a6000640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6294005ab0 tx=0x7f62940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.034+0000 7f629f400640 1 -- 172.21.15.7:0/658444008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294041070 con 0x7f62a8100b00 2024-04-20T21:40:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.034+0000 7f629f400640 1 -- 172.21.15.7:0/658444008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629402cca0 con 0x7f62a8100b00 2024-04-20T21:40:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.034+0000 7f629f400640 1 -- 172.21.15.7:0/658444008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294003d90 con 0x7f62a8100b00 2024-04-20T21:40:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.034+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a810a640 con 0x7f62a8100b00 2024-04-20T21:40:25.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.035+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a73d0 con 0x7f62a8100b00 2024-04-20T21:40:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 Processor -- start 2024-04-20T21:40:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 -- start start 2024-04-20T21:40:25.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f629f400640 1 -- 172.21.15.7:0/658444008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6294044070 con 0x7f62a8100b00 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a04f0fa050 con 0x55a04f195170 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a04f0a0490 con 0x55a04f19e670 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.038+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a04f0f6010 con 0x55a04f195a90 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39810/0 (socket says 172.21.15.7:39810) 2024-04-20T21:40:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38498/0 (socket says 172.21.15.7:38498) 2024-04-20T21:40:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 -- 172.21.15.7:0/4104396347 learned_addr learned my addr 172.21.15.7:0/4104396347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:25.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a6a00640 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 -- 172.21.15.7:0/4104396347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 msgr2=0x55a04f19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 -- 172.21.15.7:0/4104396347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 msgr2=0x55a04f1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.039+0000 7f62a7400640 1 -- 172.21.15.7:0/4104396347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04f0a8070 con 0x55a04f195170 2024-04-20T21:40:25.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f62a7400640 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62a0011790 tx=0x7f62a0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35a9508d275c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f6293e00640 1 -- 172.21.15.7:0/4104396347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0003c30 con 0x55a04f195170 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f6293e00640 1 -- 172.21.15.7:0/4104396347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a0003dd0 con 0x55a04f195170 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f6293e00640 1 -- 172.21.15.7:0/4104396347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00193e0 con 0x55a04f195170 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f629f400640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 0x7f628c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.040+0000 7f629f400640 1 -- 172.21.15.7:0/658444008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f62940f20c0 con 0x7f62a8100b00 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62a6a00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 0x7f628c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4104396347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 msgr2=0x55a04f195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f62a0011790 tx=0x7f62a0011c50 comp rx=0 tx=0).stop 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4104396347 shutdown_connections 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4104396347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f195550 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4104396347 >> 172.21.15.7:0/4104396347 conn(0x55a04f18f8a0 msgr2=0x55a04f18fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4104396347 shutdown_connections 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4104396347 wait complete. 2024-04-20T21:40:25.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.041+0000 7f62a6a00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 0x7f628c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a04f194c40 tx=0x7f6298005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 -- start start 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 Processor -- start 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f19c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.042+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a04f1a1460 con 0x55a04f195170 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a04f0b3860 con 0x55a04f19e670 2024-04-20T21:40:25.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55a04f1a15e0 con 0x55a04f195a90 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62b0a2b740 10 client.0 _ll_register_callbacks cb 0x55a04f081fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:51006/0 (socket says 172.21.15.7:51006) 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 -- 172.21.15.7:0/2413317508 learned_addr learned my addr 172.21.15.7:0/2413317508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a7400640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f19c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 -- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 msgr2=0x55a04f19c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f19c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 -- 172.21.15.7:0/2413317508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 msgr2=0x55a04f19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 -- 172.21.15.7:0/2413317508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04f0b7950 con 0x55a04f195a90 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f62a6000640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62940be170 tx=0x7f62940f56a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5114ce35958a55bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.043+0000 7f6290c00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629404b070 con 0x55a04f195a90 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f62940efa50 con 0x55a04f195a90 2024-04-20T21:40:25.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a0003dd0 con 0x55a04f195a90 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f62940ba480 con 0x55a04f195a90 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f62b0a2b740 10 client.26533 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a04f085820 con 0x55a04f195a90 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a0003dd0 con 0x55a04f195a90 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f62940de040 con 0x55a04f195a90 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f62b0a2b740 10 client.26533 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.044+0000 7f62b0a2b740 10 client.26533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:25.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62b0a2b740 20 client.26533 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f1ae560 0x55a04f1b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62b0a2b740 4 client.26533 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a04f107a70 con 0x55a04f1ae560 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62a6a00640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f1ae560 0x55a04f1b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f62a6a00640 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f1ae560 0x55a04f1b09c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de34c5af3c7039d server_cookie=7eedfd0479463b3e in_seq=0 out_seq=0 2024-04-20T21:40:25.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.045+0000 7f6290c00640 10 client.26533 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:25.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.046+0000 7f6290c00640 1 -- 172.21.15.7:0/2413317508 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a04f107a70 con 0x55a04f1ae560 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.046+0000 7f6290c00640 10 client.26533 handle_command_reply: tid=0 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.046+0000 7f62b0a2b740 1 client.26533 shutdown 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.046+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 msgr2=0x55a04f19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.046+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f62940be170 tx=0x7f62940f56a0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 shutdown_connections 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f19c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f1ae560 0x55a04f1b09c0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195a90 0x55a04f19bbf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2413317508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195170 0x55a04f19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f6290c00640 0 client.26533 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 >> 172.21.15.7:0/2413317508 conn(0x55a04f18f8a0 msgr2=0x55a04f1915f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 shutdown_connections 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2413317508 wait complete. 2024-04-20T21:40:25.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.047+0000 7f62b0a2b740 20 client.26533 trim_cache size 0 max 16384 2024-04-20T21:40:25.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 Processor -- start 2024-04-20T21:40:25.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 -- start start 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1b6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81b25a0 con 0x55a04f195a90 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81afe60 con 0x55a04f19e670 2024-04-20T21:40:25.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.062+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81adcd0 con 0x55a04f195170 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39826/0 (socket says 172.21.15.7:39826) 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 -- 172.21.15.7:0/4026834603 learned_addr learned my addr 172.21.15.7:0/4026834603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6a00640 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 -- 172.21.15.7:0/4026834603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 msgr2=0x55a04f1aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 -- 172.21.15.7:0/4026834603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 msgr2=0x55a04f1b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 -- 172.21.15.7:0/4026834603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04ee0c3f0 con 0x55a04f195a90 2024-04-20T21:40:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a7400640 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.063+0000 7f62a6000640 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f62940c6e10 tx=0x7f629402bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ea430174048183 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.064+0000 7f6293400640 1 -- 172.21.15.7:0/4026834603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629404b070 con 0x55a04f195a90 2024-04-20T21:40:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.064+0000 7f6293400640 1 -- 172.21.15.7:0/4026834603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294004c80 con 0x55a04f195a90 2024-04-20T21:40:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.064+0000 7f6293400640 1 -- 172.21.15.7:0/4026834603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62940bdb40 con 0x55a04f195a90 2024-04-20T21:40:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.064+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4026834603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 msgr2=0x55a04f19dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.064+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f62940c6e10 tx=0x7f629402bc40 comp rx=0 tx=0).stop 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4026834603 shutdown_connections 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f19dd70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/4026834603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1aa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4026834603 >> 172.21.15.7:0/4026834603 conn(0x55a04f18f8a0 msgr2=0x55a04f1969d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4026834603 shutdown_connections 2024-04-20T21:40:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 -- 172.21.15.7:0/4026834603 wait complete. 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 -- start start 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.065+0000 7f62b0a2b740 1 Processor -- start 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81ae320 con 0x55a04f195a90 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81af790 con 0x55a04f19e670 2024-04-20T21:40:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81ae9f0 con 0x55a04f195170 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62b0a2b740 10 client.0 _ll_register_callbacks cb 0x7f6294003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39836/0 (socket says 172.21.15.7:39836) 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 -- 172.21.15.7:0/2822362966 learned_addr learned my addr 172.21.15.7:0/2822362966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a7400640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6a00640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 -- 172.21.15.7:0/2822362966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 msgr2=0x55a04f1a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 -- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 msgr2=0x55a04f1a2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a6000640 1 -- 172.21.15.7:0/2822362966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04eed67d0 con 0x55a04f195a90 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.066+0000 7f62a7400640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f62a6000640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f629402e040 tx=0x7f62940c6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f78025be2199f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f6290c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62940d7060 con 0x55a04f195a90 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f62940d2bd0 con 0x55a04f195a90 2024-04-20T21:40:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6294004c80 con 0x55a04f195a90 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f629404b070 con 0x55a04f195a90 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f62b0a2b740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.067+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a04eed67d0 con 0x55a04f195a90 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294004c80 con 0x55a04f195a90 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f62940de040 con 0x55a04f195a90 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 20 client.16377 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f215020 0x55a04f217440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 4 client.16377 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a04f1018e0 con 0x55a04f215020 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.068+0000 7f62a6a00640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f215020 0x55a04f217440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62a6a00640 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f215020 0x55a04f217440 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a9c84b4c92e44d server_cookie=dda5d054eb8d2b6a in_seq=0 out_seq=0 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f6290c00640 10 client.16377 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f6290c00640 1 -- 172.21.15.7:0/2822362966 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a04f1018e0 con 0x55a04f215020 2024-04-20T21:40:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f6290c00640 10 client.16377 handle_command_reply: tid=0 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 client.16377 shutdown 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 msgr2=0x55a04f1a2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f629402e040 tx=0x7f62940c6bc0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 shutdown_connections 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a04f19e670 0x55a04f1a2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55a04f215020 0x55a04f217440 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a04f195a90 0x55a04f1a2830 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 --2- 172.21.15.7:0/2822362966 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a04f195170 0x55a04f1a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 >> 172.21.15.7:0/2822362966 conn(0x55a04f18f8a0 msgr2=0x55a04f18e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f6290c00640 0 client.16377 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.069+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 shutdown_connections 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62b0a2b740 1 -- 172.21.15.7:0/2822362966 wait complete. 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62b0a2b740 20 client.16377 trim_cache size 0 max 16384 2024-04-20T21:40:25.076 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 msgr2=0x7f628c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 0x7f628c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55a04f194c40 tx=0x7f6298005f90 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 msgr2=0x7f62a81a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6294005ab0 tx=0x7f62940042f0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 shutdown_connections 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f628c0a3d20 0x7f628c0a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62a810ae70 0x7f62a8109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8100b00 0x7f62a81a0be0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 --2- 172.21.15.7:0/658444008 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62a8068580 0x7f62a81a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.070+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 >> 172.21.15.7:0/658444008 conn(0x7f62a8077dd0 msgr2=0x7f62a810c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.071+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 shutdown_connections 2024-04-20T21:40:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:25.071+0000 7f62a7e00640 1 -- 172.21.15.7:0/658444008 wait complete. 2024-04-20T21:40:26.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:40:26.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:40:26.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 -- 172.21.15.7:0/3670887197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac108e40 msgr2=0x7f3cac115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 --2- 172.21.15.7:0/3670887197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac108e40 0x7f3cac115210 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009ca0 tx=0x7f3ca402c100 comp rx=0 tx=0).stop 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 -- 172.21.15.7:0/3670887197 shutdown_connections 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 --2- 172.21.15.7:0/3670887197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac108e40 0x7f3cac115210 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 --2- 172.21.15.7:0/3670887197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac1084a0 0x7f3cac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 --2- 172.21.15.7:0/3670887197 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 0x7f3cac10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 -- 172.21.15.7:0/3670887197 >> 172.21.15.7:0/3670887197 conn(0x7f3cac077d70 msgr2=0x7f3cac106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 -- 172.21.15.7:0/3670887197 shutdown_connections 2024-04-20T21:40:26.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.322+0000 7f3cb2800640 1 -- 172.21.15.7:0/3670887197 wait complete. 2024-04-20T21:40:26.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.323+0000 7f3cb2800640 1 Processor -- start 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.323+0000 7f3cb2800640 1 -- start start 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac1084a0 0x7f3cac1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 0x7f3cac19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac118710 con 0x7f3cac108e40 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac118590 con 0x7f3cac1084a0 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb2800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac118890 con 0x7f3cac10e3e0 2024-04-20T21:40:26.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39848/0 (socket says 172.21.15.7:39848) 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb0a00640 1 -- 172.21.15.7:0/2070155868 learned_addr learned my addr 172.21.15.7:0/2070155868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 0x7f3cac19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.324+0000 7f3cb1400640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac1084a0 0x7f3cac1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 -- 172.21.15.7:0/2070155868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 msgr2=0x7f3cac19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 0x7f3cac19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 -- 172.21.15.7:0/2070155868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac1084a0 msgr2=0x7f3cac1a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac1084a0 0x7f3cac1a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 -- 172.21.15.7:0/2070155868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac19f1d0 con 0x7f3cac108e40 2024-04-20T21:40:26.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e9f0 tx=0x7f3c9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.325+0000 7f3caa000640 1 -- 172.21.15.7:0/2070155868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800cd80 con 0x7f3cac108e40 2024-04-20T21:40:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.326+0000 7f3caa000640 1 -- 172.21.15.7:0/2070155868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c98010910 con 0x7f3cac108e40 2024-04-20T21:40:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.326+0000 7f3caa000640 1 -- 172.21.15.7:0/2070155868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98018990 con 0x7f3cac108e40 2024-04-20T21:40:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.326+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac19f4c0 con 0x7f3cac108e40 2024-04-20T21:40:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.326+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1ab710 con 0x7f3cac108e40 2024-04-20T21:40:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.329+0000 7f3cb6b63740 1 Processor -- start 2024-04-20T21:40:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.329+0000 7f3caa000640 1 -- 172.21.15.7:0/2070155868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3c98010ac0 con 0x7f3cac108e40 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.329+0000 7f3cb6b63740 1 -- start start 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a70f0 0x556ebc2a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 0x556ebc2b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ebc20bd20 con 0x556ebc2b05f0 2024-04-20T21:40:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ebc1b23e0 con 0x556ebc2a70f0 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556ebc207ce0 con 0x556ebc2a7a10 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 0x556ebc2b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a70f0 0x556ebc2a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 0x556ebc2b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39864/0 (socket says 172.21.15.7:39864) 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb0a00640 1 -- 172.21.15.7:0/784212771 learned_addr learned my addr 172.21.15.7:0/784212771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.330+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 -- 172.21.15.7:0/784212771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a70f0 msgr2=0x556ebc2a74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a70f0 0x556ebc2a74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 -- 172.21.15.7:0/784212771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 msgr2=0x556ebc2b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 0x556ebc2b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 -- 172.21.15.7:0/784212771 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ebc1d1980 con 0x556ebc2a7a10 2024-04-20T21:40:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3caa000640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 0x7f3c940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3cb1400640 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b00b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009bd0 tx=0x7f3ca0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d76c9ec6daa927fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.331+0000 7f3caa000640 1 -- 172.21.15.7:0/2070155868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f3c980d0b60 con 0x7f3cac108e40 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 0x7f3c940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3c9e600640 1 -- 172.21.15.7:0/784212771 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004050 con 0x556ebc2a7a10 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3c9e600640 1 -- 172.21.15.7:0/784212771 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca0005500 con 0x556ebc2a7a10 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3c9e600640 1 -- 172.21.15.7:0/784212771 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0004d40 con 0x556ebc2a7a10 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3cb6b63740 1 -- 172.21.15.7:0/784212771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 msgr2=0x556ebc2b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b00b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0009bd0 tx=0x7f3ca0011810 comp rx=0 tx=0).stop 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.332+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 0x7f3c940a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4005e30 tx=0x7f3ca4005d80 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 -- 172.21.15.7:0/784212771 shutdown_connections 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2b05f0 0x556ebc2b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b00b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/784212771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a70f0 0x556ebc2a74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 -- 172.21.15.7:0/784212771 >> 172.21.15.7:0/784212771 conn(0x556ebc2a1820 msgr2=0x556ebc2a1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 -- 172.21.15.7:0/784212771 shutdown_connections 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 -- 172.21.15.7:0/784212771 wait complete. 2024-04-20T21:40:26.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 -- start start 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.333+0000 7f3cb6b63740 1 Processor -- start 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc2ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ebc2b33e0 con 0x556ebc2a7a10 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ebc1c57a0 con 0x556ebc2b05f0 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556ebc2b3560 con 0x556ebc2a70f0 2024-04-20T21:40:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb6b63740 10 client.0 _ll_register_callbacks cb 0x556ebc1c7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39866/0 (socket says 172.21.15.7:39866) 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:51074/0 (socket says 172.21.15.7:51074) 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb0a00640 1 -- 172.21.15.7:0/4272857168 learned_addr learned my addr 172.21.15.7:0/4272857168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.334+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc2ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 -- 172.21.15.7:0/4272857168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 msgr2=0x556ebc2ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 -- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 msgr2=0x556ebc2ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc2ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 -- 172.21.15.7:0/4272857168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ebbfe67e0 con 0x556ebc2a7a10 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb1400640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c98098fd0 tx=0x7f3c98098ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbb6b56541b0f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3c92a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9809cb70 con 0x556ebc2a7a10 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f3c9809ae70 con 0x556ebc2a7a10 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca0005500 con 0x556ebc2a7a10 2024-04-20T21:40:26.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3c980a0e70 con 0x556ebc2a7a10 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb6b63740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.335+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ebc046f70 con 0x556ebc2a7a10 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca0005500 con 0x556ebc2a7a10 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3c9809bfc0 con 0x556ebc2a7a10 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 20 client.16389 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:26.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc2c0920 0x556ebc2c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 4 client.16389 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.336+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ebc219740 con 0x556ebc2c0920 2024-04-20T21:40:26.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.337+0000 7f3cb1400640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc2c0920 0x556ebc2c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.337+0000 7f3cb1400640 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc2c0920 0x556ebc2c2d90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44091367cf8edecd server_cookie=a15a7ee8731c19c2 in_seq=0 out_seq=0 2024-04-20T21:40:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.337+0000 7f3c92a00640 10 client.16389 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3c92a00640 1 -- 172.21.15.7:0/4272857168 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ebc219740 con 0x556ebc2c0920 2024-04-20T21:40:26.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3c92a00640 10 client.16389 handle_command_reply: tid=0 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 client.16389 shutdown 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 msgr2=0x556ebc2adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c98098fd0 tx=0x7f3c98098ed0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 shutdown_connections 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc2ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc2c0920 0x556ebc2c2d90 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a7a10 0x556ebc2adbc0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/4272857168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a70f0 0x556ebc2ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3c92a00640 0 client.16389 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.338+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 >> 172.21.15.7:0/4272857168 conn(0x556ebc2a1820 msgr2=0x556ebc2a3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.339+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 shutdown_connections 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.339+0000 7f3cb6b63740 1 -- 172.21.15.7:0/4272857168 wait complete. 2024-04-20T21:40:26.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.339+0000 7f3cb6b63740 20 client.16389 trim_cache size 0 max 16384 2024-04-20T21:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.359+0000 7f3cb6b63740 1 Processor -- start 2024-04-20T21:40:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.359+0000 7f3cb6b63740 1 -- start start 2024-04-20T21:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.359+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.359+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a7a10 0x556ebc2bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.359+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b6940 con 0x556ebc2a70f0 2024-04-20T21:40:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b4200 con 0x556ebc2a7a10 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b2070 con 0x556ebc2b05f0 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:51084/0 (socket says 172.21.15.7:51084) 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1e00640 1 -- 172.21.15.7:0/3883377743 learned_addr learned my addr 172.21.15.7:0/3883377743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a7a10 0x556ebc2bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1400640 1 -- 172.21.15.7:0/3883377743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 msgr2=0x556ebc2c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1400640 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1400640 1 -- 172.21.15.7:0/3883377743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a7a10 msgr2=0x556ebc2bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.360+0000 7f3cb1400640 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a7a10 0x556ebc2bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3cb1400640 1 -- 172.21.15.7:0/3883377743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ebbede170 con 0x556ebc2a70f0 2024-04-20T21:40:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3cb1400640 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2afd40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001b040 tx=0x7f3ca00049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3f4ec1cd9b61f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3c9dc00640 1 -- 172.21.15.7:0/3883377743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0033070 con 0x556ebc2a70f0 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3c9dc00640 1 -- 172.21.15.7:0/3883377743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca002e8c0 con 0x556ebc2a70f0 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.361+0000 7f3c9dc00640 1 -- 172.21.15.7:0/3883377743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00035f0 con 0x556ebc2a70f0 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 -- 172.21.15.7:0/3883377743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 msgr2=0x556ebc2afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2afd40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001b040 tx=0x7f3ca00049a0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 -- 172.21.15.7:0/3883377743 shutdown_connections 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2a7a10 0x556ebc2bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2b05f0 0x556ebc2c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/3883377743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2afd40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 -- 172.21.15.7:0/3883377743 >> 172.21.15.7:0/3883377743 conn(0x556ebc2a1820 msgr2=0x556ebc2a34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 -- 172.21.15.7:0/3883377743 shutdown_connections 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.362+0000 7f3cb6b63740 1 -- 172.21.15.7:0/3883377743 wait complete. 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 -- start start 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 Processor -- start 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc31fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b26c0 con 0x556ebc2a70f0 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b3b30 con 0x556ebc2b05f0 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b2d90 con 0x556ebc2a7a10 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.363+0000 7f3cb6b63740 10 client.0 _ll_register_callbacks cb 0x7f3c980d4300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39882/0 (socket says 172.21.15.7:39882) 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 -- 172.21.15.7:0/2991181432 learned_addr learned my addr 172.21.15.7:0/2991181432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1e00640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc31fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 -- 172.21.15.7:0/2991181432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 msgr2=0x556ebc2b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 -- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 msgr2=0x556ebc31fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc31fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb1400640 1 -- 172.21.15.7:0/2991181432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ebc320490 con 0x556ebc2a70f0 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.364+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3cb1400640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ca002ecb0 tx=0x7f3ca0005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7a88c30f6908e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3c92a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca003b020 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f3ca0004360 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca002e8c0 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3ca0046050 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3cb6b63740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.365+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ebc327850 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca002e8c0 con 0x556ebc2a70f0 2024-04-20T21:40:26.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3ca0036070 con 0x556ebc2a70f0 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 20 client.16395 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc327e30 0x556ebc32a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 4 client.16395 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:40:26.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556ebc2135b0 con 0x556ebc327e30 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.366+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc327e30 0x556ebc32a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.367+0000 7f3cb0a00640 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc327e30 0x556ebc32a250 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d5c88ea96d2e61 server_cookie=479b8e804ce4010c in_seq=0 out_seq=0 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.367+0000 7f3c92a00640 10 client.16395 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3c92a00640 1 -- 172.21.15.7:0/2991181432 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556ebc2135b0 con 0x556ebc327e30 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3c92a00640 10 client.16395 handle_command_reply: tid=0 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 client.16395 shutdown 2024-04-20T21:40:26.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 msgr2=0x556ebc2b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3ca002ecb0 tx=0x7f3ca0005fc0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 shutdown_connections 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ebc2b05f0 0x556ebc31fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.368+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x556ebc327e30 0x556ebc32a250 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556ebc2a7a10 0x556ebc2b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 1 --2- 172.21.15.7:0/2991181432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ebc2a70f0 0x556ebc2b4a00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3c92a00640 0 client.16395 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 >> 172.21.15.7:0/2991181432 conn(0x556ebc2a1820 msgr2=0x556ebc2ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 shutdown_connections 2024-04-20T21:40:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 1 -- 172.21.15.7:0/2991181432 wait complete. 2024-04-20T21:40:26.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.369+0000 7f3cb6b63740 20 client.16395 trim_cache size 0 max 16384 2024-04-20T21:40:26.374 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:40:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "fa44e48d-6d08-4c1d-a58d-914d4defbac3", 2024-04-20T21:40:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:40:26.375 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:26.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 msgr2=0x7f3c940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 0x7f3c940a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4005e30 tx=0x7f3ca4005d80 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 msgr2=0x7f3cac1a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800e9f0 tx=0x7f3c9800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 shutdown_connections 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3cac10e3e0 0x7f3cac19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cac108e40 0x7f3cac1a4d90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3c940a3d20 0x7f3c940a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 --2- 172.21.15.7:0/2070155868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cac1084a0 0x7f3cac1a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 >> 172.21.15.7:0/2070155868 conn(0x7f3cac077d70 msgr2=0x7f3cac078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 shutdown_connections 2024-04-20T21:40:26.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.370+0000 7f3cb2800640 1 -- 172.21.15.7:0/2070155868 wait complete. 2024-04-20T21:40:26.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:26.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:26.454 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:26 smithi007 ceph-mon[16867]: pgmap v218: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:40:26.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 -- 172.21.15.7:0/37148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f0100b30 msgr2=0x7f18f0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 --2- 172.21.15.7:0/37148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f0100b30 0x7f18f0100f10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009900 tx=0x7f18e002bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 -- 172.21.15.7:0/37148116 shutdown_connections 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 --2- 172.21.15.7:0/37148116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01014e0 0x7f18f010e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 --2- 172.21.15.7:0/37148116 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f0110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 --2- 172.21.15.7:0/37148116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f0100b30 0x7f18f0100f10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 -- 172.21.15.7:0/37148116 >> 172.21.15.7:0/37148116 conn(0x7f18f00fc7a0 msgr2=0x7f18f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 -- 172.21.15.7:0/37148116 shutdown_connections 2024-04-20T21:40:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.592+0000 7f18ef200640 1 -- 172.21.15.7:0/37148116 wait complete. 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.593+0000 7f18ef200640 1 Processor -- start 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 -- start start 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 0x7f18f01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 0x7f18f01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0114370 con 0x7f18f01014e0 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01141f0 con 0x7f18f0100b30 2024-04-20T21:40:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ef200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01144f0 con 0x7f18f010e6e0 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 0x7f18f01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ee800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 0x7f18f01a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39904/0 (socket says 172.21.15.7:39904) 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 0x7f18f01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.594+0000 7f18e5400640 1 -- 172.21.15.7:0/866537890 learned_addr learned my addr 172.21.15.7:0/866537890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 -- 172.21.15.7:0/866537890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 msgr2=0x7f18f01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 0x7f18f01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 -- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 msgr2=0x7f18f01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 0x7f18f01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 -- 172.21.15.7:0/866537890 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f019b010 con 0x7f18f010e6e0 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ede00640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 0x7f18f01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18ee800640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f019a750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18f0101a80 tx=0x7f18e800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.595+0000 7f18e7400640 1 -- 172.21.15.7:0/866537890 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800cdf0 con 0x7f18f010e6e0 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.596+0000 7f18e7400640 1 -- 172.21.15.7:0/866537890 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e8005020 con 0x7f18f010e6e0 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.596+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f019b300 con 0x7f18f010e6e0 2024-04-20T21:40:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.596+0000 7f18e7400640 1 -- 172.21.15.7:0/866537890 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e8010670 con 0x7f18f010e6e0 2024-04-20T21:40:26.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.596+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a75c0 con 0x7f18f010e6e0 2024-04-20T21:40:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.599+0000 7f18f7bb4740 1 Processor -- start 2024-04-20T21:40:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.599+0000 7f18e7400640 1 -- 172.21.15.7:0/866537890 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f18e80054d0 con 0x7f18f010e6e0 2024-04-20T21:40:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.599+0000 7f18f7bb4740 1 -- start start 2024-04-20T21:40:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e69340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e6bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa39dc6150 con 0x55fa39e60ca0 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fa39d6c370 con 0x55fa39e60380 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa39dc2110 con 0x55fa39e69880 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18e5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18e5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38634/0 (socket says 172.21.15.7:38634) 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18e5400640 1 -- 172.21.15.7:0/3139112638 learned_addr learned my addr 172.21.15.7:0/3139112638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18e5400640 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e6bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.600+0000 7f18ede00640 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e69340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 -- 172.21.15.7:0/3139112638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 msgr2=0x55fa39e6bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e6bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 -- 172.21.15.7:0/3139112638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 msgr2=0x55fa39e69340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e7400640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e69340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 -- 172.21.15.7:0/3139112638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39ba27d0 con 0x55fa39e60380 2024-04-20T21:40:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e7400640 1 -- 172.21.15.7:0/866537890 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f18e80d0e20 con 0x7f18f010e6e0 2024-04-20T21:40:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18e5400640 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f18d8011910 tx=0x7f18d8009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385fe3b45d3deccd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18ee800640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18df400640 1 -- 172.21.15.7:0/3139112638 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800f7c0 con 0x55fa39e60380 2024-04-20T21:40:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18df400640 1 -- 172.21.15.7:0/3139112638 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d800fe40 con 0x55fa39e60380 2024-04-20T21:40:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.601+0000 7f18df400640 1 -- 172.21.15.7:0/3139112638 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d800e3b0 con 0x55fa39e60380 2024-04-20T21:40:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18ee800640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 0x7f18d00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18e80a54c0 tx=0x7f18e80058a0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 -- 172.21.15.7:0/3139112638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 msgr2=0x55fa39e60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f18d8011910 tx=0x7f18d8009540 comp rx=0 tx=0).stop 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 -- 172.21.15.7:0/3139112638 shutdown_connections 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e6bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e69340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/3139112638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e60760 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 -- 172.21.15.7:0/3139112638 >> 172.21.15.7:0/3139112638 conn(0x55fa39e5aab0 msgr2=0x55fa39e5ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 -- 172.21.15.7:0/3139112638 shutdown_connections 2024-04-20T21:40:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.602+0000 7f18f7bb4740 1 -- 172.21.15.7:0/3139112638 wait complete. 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 -- start start 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 Processor -- start 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e668a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39e67320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa39e6c670 con 0x55fa39e69880 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fa39d7f740 con 0x55fa39e60ca0 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55fa39e6c7f0 con 0x55fa39e60380 2024-04-20T21:40:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.603+0000 7f18f7bb4740 10 client.0 _ll_register_callbacks cb 0x55fa39d7ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18e5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38638/0 (socket says 172.21.15.7:38638) 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 -- 172.21.15.7:0/4098281153 learned_addr learned my addr 172.21.15.7:0/4098281153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18e5400640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39e67320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 -- 172.21.15.7:0/4098281153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 msgr2=0x55fa39e668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e668a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 -- 172.21.15.7:0/4098281153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 msgr2=0x55fa39e67320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39e67320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 -- 172.21.15.7:0/4098281153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa39ad83f0 con 0x55fa39e60ca0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18e5400640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39e67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18ede00640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f18e002c170 tx=0x7f18e002c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d34ae24469082e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.604+0000 7f18d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e002cb70 con 0x55fa39e60ca0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f18e0004040 con 0x55fa39e60ca0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d800fe40 con 0x55fa39e60ca0 2024-04-20T21:40:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18e0004240 con 0x55fa39e60ca0 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18f7bb4740 10 client.26615 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa39a980f0 con 0x55fa39e60ca0 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.605+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d800fe40 con 0x55fa39e60ca0 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f18e004e070 con 0x55fa39e60ca0 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 10 client.26615 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 10 client.26615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 20 client.26615 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39e79c30 0x55fa39e7c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 4 client.26615 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa39dd3b70 con 0x55fa39e79c30 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.606+0000 7f18e5400640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39e79c30 0x55fa39e7c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.607+0000 7f18e5400640 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39e79c30 0x55fa39e7c070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6b7372b124b121 server_cookie=cba4902593583cb7 in_seq=0 out_seq=0 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.607+0000 7f18d7e00640 10 client.26615 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.607+0000 7f18d7e00640 1 -- 172.21.15.7:0/4098281153 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa39dd3b70 con 0x55fa39e79c30 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.607+0000 7f18d7e00640 10 client.26615 handle_command_reply: tid=0 2024-04-20T21:40:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.607+0000 7f18f7bb4740 1 client.26615 shutdown 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 msgr2=0x55fa39e66de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f18e002c170 tx=0x7f18e002c880 comp rx=0 tx=0).stop 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 shutdown_connections 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e66de0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39e79c30 0x55fa39e7c070 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39e67320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/4098281153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18d7e00640 0 client.26615 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 >> 172.21.15.7:0/4098281153 conn(0x55fa39e5aab0 msgr2=0x55fa39e5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 shutdown_connections 2024-04-20T21:40:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 1 -- 172.21.15.7:0/4098281153 wait complete. 2024-04-20T21:40:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.608+0000 7f18f7bb4740 20 client.26615 trim_cache size 0 max 16384 2024-04-20T21:40:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.627+0000 7f18f7bb4740 1 Processor -- start 2024-04-20T21:40:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 -- start start 2024-04-20T21:40:26.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e61e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01b25d0 con 0x55fa39e60ca0 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01afe90 con 0x55fa39e60380 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01add00 con 0x55fa39e69880 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39924/0 (socket says 172.21.15.7:39924) 2024-04-20T21:40:26.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18ede00640 1 -- 172.21.15.7:0/1555459843 learned_addr learned my addr 172.21.15.7:0/1555459843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18e5400640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.628+0000 7f18e5400640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e61e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 -- 172.21.15.7:0/1555459843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 msgr2=0x55fa39e61e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e61e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 -- 172.21.15.7:0/1555459843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 msgr2=0x55fa39e657e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e657e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 -- 172.21.15.7:0/1555459843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39e5a220 con 0x55fa39e60ca0 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18e5400640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e61e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:26.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18ede00640 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f18e002bc70 tx=0x7f18e0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf68cba814853bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18dea00640 1 -- 172.21.15.7:0/1555459843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0054070 con 0x55fa39e60ca0 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18dea00640 1 -- 172.21.15.7:0/1555459843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0003fa0 con 0x55fa39e60ca0 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.629+0000 7f18dea00640 1 -- 172.21.15.7:0/1555459843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00051a0 con 0x55fa39e60ca0 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- 172.21.15.7:0/1555459843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 msgr2=0x55fa39e75530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f18e002bc70 tx=0x7f18e0005b50 comp rx=0 tx=0).stop 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- 172.21.15.7:0/1555459843 shutdown_connections 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e69880 0x55fa39e61e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e60ca0 0x55fa39e75530 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/1555459843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60380 0x55fa39e657e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- 172.21.15.7:0/1555459843 >> 172.21.15.7:0/1555459843 conn(0x55fa39e5aab0 msgr2=0x55fa39e59b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- 172.21.15.7:0/1555459843 shutdown_connections 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- 172.21.15.7:0/1555459843 wait complete. 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.630+0000 7f18f7bb4740 1 -- start start 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 Processor -- start 2024-04-20T21:40:26.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e63c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39edf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ae350 con 0x55fa39e69880 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01af7c0 con 0x55fa39e60ca0 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01aea20 con 0x55fa39e60380 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18f7bb4740 10 client.0 _ll_register_callbacks cb 0x7f18e80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.631+0000 7f18e5400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e63c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39edf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:38652/0 (socket says 172.21.15.7:38652) 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 -- 172.21.15.7:0/687710508 learned_addr learned my addr 172.21.15.7:0/687710508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 -- 172.21.15.7:0/687710508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 msgr2=0x55fa39e63c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e63c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 -- 172.21.15.7:0/687710508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 msgr2=0x55fa39edf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39edf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 -- 172.21.15.7:0/687710508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa39edf6f0 con 0x55fa39e60ca0 2024-04-20T21:40:26.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18e5400640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39edf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18ede00640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009880 tx=0x7f18e0004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c8791d4bef91a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.632+0000 7f18d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0054040 con 0x55fa39e60ca0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f18e002cc90 con 0x55fa39e60ca0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e0003fa0 con 0x55fa39e60ca0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18e00044a0 con 0x55fa39e60ca0 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 10 client.26621 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:26.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa39ee0970 con 0x55fa39e60ca0 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0003fa0 con 0x55fa39e60ca0 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f18e004e070 con 0x55fa39e60ca0 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 10 client.26621 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.633+0000 7f18f7bb4740 20 client.26621 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39ee0e80 0x55fa39ee3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18f7bb4740 4 client.26621 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:26.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa39dcd9e0 con 0x55fa39ee0e80 2024-04-20T21:40:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18e5400640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39ee0e80 0x55fa39ee3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18e5400640 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39ee0e80 0x55fa39ee3270 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad86d19ea63f7280 server_cookie=79be66b79f9e708 in_seq=0 out_seq=0 2024-04-20T21:40:26.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.634+0000 7f18d7e00640 10 client.26621 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18d7e00640 1 -- 172.21.15.7:0/687710508 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fa39dcd9e0 con 0x55fa39ee0e80 2024-04-20T21:40:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18d7e00640 10 client.26621 handle_command_reply: tid=0 2024-04-20T21:40:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 client.26621 shutdown 2024-04-20T21:40:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 msgr2=0x55fa39e64170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009880 tx=0x7f18e0004150 comp rx=0 tx=0).stop 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 shutdown_connections 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fa39e60ca0 0x55fa39e64170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55fa39ee0e80 0x55fa39ee3270 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa39e69880 0x55fa39edf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 --2- 172.21.15.7:0/687710508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55fa39e60380 0x55fa39e63c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18d7e00640 0 client.26621 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 >> 172.21.15.7:0/687710508 conn(0x55fa39e5aab0 msgr2=0x55fa39e5bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 shutdown_connections 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 1 -- 172.21.15.7:0/687710508 wait complete. 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.635+0000 7f18f7bb4740 20 client.26621 trim_cache size 0 max 16384 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:40:26.642 INFO:teuthology.orchestra.run.smithi007.stdout: "fa44e48d-6d08-4c1d-a58d-914d4defbac3": { 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "fa44e48d-6d08-4c1d-a58d-914d4defbac3", 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:40:26.643 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:26.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 msgr2=0x7f18d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 0x7f18d00a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18e80a54c0 tx=0x7f18e80058a0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 msgr2=0x7f18f019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:26.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f019a750 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18f0101a80 tx=0x7f18e800ef10 comp rx=0 tx=0).stop 2024-04-20T21:40:26.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 shutdown_connections 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f18f010e6e0 0x7f18f019a750 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18f01014e0 0x7f18f01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f18d00a3d20 0x7f18d00a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 --2- 172.21.15.7:0/866537890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0100b30 0x7f18f01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 >> 172.21.15.7:0/866537890 conn(0x7f18f00fc7a0 msgr2=0x7f18f00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 shutdown_connections 2024-04-20T21:40:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:26.636+0000 7f18ef200640 1 -- 172.21.15.7:0/866537890 wait complete. 2024-04-20T21:40:26.652 INFO:tasks.cephfs.filesystem:scrub status for tag:fa44e48d-6d08-4c1d-a58d-914d4defbac3 - {'path': '/', 'tag': 'fa44e48d-6d08-4c1d-a58d-914d4defbac3', 'options': 'recursive,force'} 2024-04-20T21:40:26.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:26 smithi016 ceph-mon[21409]: pgmap v218: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:40:26.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:26 smithi186 ceph-mon[22157]: pgmap v218: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:40:27.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:27 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:40:27.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:27 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:40:27.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:27.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:27 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:28.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:28 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:40:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:28 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:28.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:28 smithi016 ceph-mon[21409]: pgmap v219: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:28 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:40:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:28 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:28.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:28 smithi186 ceph-mon[22157]: pgmap v219: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:28.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:28 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:40:28.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:28 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:40:28.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:28 smithi007 ceph-mon[16867]: pgmap v219: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:30.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:30 smithi016 ceph-mon[21409]: pgmap v220: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-04-20T21:40:30.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:30 smithi186 ceph-mon[22157]: pgmap v220: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-04-20T21:40:30.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:30 smithi007 ceph-mon[16867]: pgmap v220: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 7 op/s 2024-04-20T21:40:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:32 smithi016 ceph-mon[21409]: pgmap v221: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-20T21:40:32.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:32 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:32.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:32 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:32.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:32 smithi186 ceph-mon[22157]: pgmap v221: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-20T21:40:32.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:32 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:32.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:32 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:32 smithi007 ceph-mon[16867]: pgmap v221: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2024-04-20T21:40:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:32 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:32 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:33 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:40:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:33 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:40:33.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:33 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:40:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:34 smithi016 ceph-mon[21409]: pgmap v222: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:34 smithi186 ceph-mon[22157]: pgmap v222: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:34.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:34 smithi007 ceph-mon[16867]: pgmap v222: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:36 smithi016 ceph-mon[21409]: pgmap v223: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:36 smithi186 ceph-mon[22157]: pgmap v223: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:36 smithi007 ceph-mon[16867]: pgmap v223: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-20T21:40:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:38 smithi016 ceph-mon[21409]: pgmap v224: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-04-20T21:40:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:38 smithi186 ceph-mon[22157]: pgmap v224: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-04-20T21:40:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:38 smithi007 ceph-mon[16867]: pgmap v224: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-04-20T21:40:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:40 smithi016 ceph-mon[21409]: pgmap v225: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-20T21:40:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:40 smithi186 ceph-mon[22157]: pgmap v225: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-20T21:40:40.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:40 smithi007 ceph-mon[16867]: pgmap v225: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-20T21:40:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:42 smithi016 ceph-mon[21409]: pgmap v226: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:40:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:42 smithi186 ceph-mon[22157]: pgmap v226: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:40:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:42 smithi007 ceph-mon[16867]: pgmap v226: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:40:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:44 smithi016 ceph-mon[21409]: pgmap v227: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:40:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:44 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:44 smithi186 ceph-mon[22157]: pgmap v227: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:40:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:44 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:44.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:44 smithi007 ceph-mon[16867]: pgmap v227: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:40:44.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:44 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:40:45.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:45 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:45 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:45.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:45 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:40:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:45 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:45 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:45 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:40:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:45 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:40:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:45 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:40:45.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:45 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:40:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:46 smithi016 ceph-mon[21409]: pgmap v228: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:40:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:46 smithi186 ceph-mon[22157]: pgmap v228: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:40:46.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:46 smithi007 ceph-mon[16867]: pgmap v228: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:40:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:48 smithi016 ceph-mon[21409]: pgmap v229: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:48 smithi186 ceph-mon[22157]: pgmap v229: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:48 smithi007 ceph-mon[16867]: pgmap v229: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:50.697 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:50 smithi016 ceph-mon[21409]: pgmap v230: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:40:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:50 smithi186 ceph-mon[22157]: pgmap v230: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:40:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:50 smithi007 ceph-mon[16867]: pgmap v230: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:40:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:52 smithi016 ceph-mon[21409]: pgmap v231: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:52 smithi186 ceph-mon[22157]: pgmap v231: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:52 smithi007 ceph-mon[16867]: pgmap v231: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:54 smithi016 ceph-mon[21409]: pgmap v232: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:40:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:54 smithi186 ceph-mon[22157]: pgmap v232: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:40:54.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:54 smithi007 ceph-mon[16867]: pgmap v232: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:40:56.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:56.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:56 smithi016 ceph-mon[21409]: pgmap v233: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:40:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:56 smithi186 ceph-mon[22157]: pgmap v233: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:40:56.737 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:56 smithi007 ceph-mon[16867]: pgmap v233: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:40:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 -- 172.21.15.7:0/2571828564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb94104b50 msgr2=0x7fdb94108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/2571828564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb94104b50 0x7fdb94108b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009900 tx=0x7fdb8402bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 -- 172.21.15.7:0/2571828564 shutdown_connections 2024-04-20T21:40:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/2571828564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb94104b50 0x7fdb94108b90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/2571828564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb94109270 0x7fdb94110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/2571828564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb941041a0 0x7fdb94104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.882+0000 7fdb9ac00640 1 -- 172.21.15.7:0/2571828564 >> 172.21.15.7:0/2571828564 conn(0x7fdb940ffa40 msgr2=0x7fdb94101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.883+0000 7fdb9ac00640 1 -- 172.21.15.7:0/2571828564 shutdown_connections 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.883+0000 7fdb9ac00640 1 -- 172.21.15.7:0/2571828564 wait complete. 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 Processor -- start 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 -- start start 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb94109270 0x7fdb941a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 0x7fdb9419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94114350 con 0x7fdb94109270 2024-04-20T21:40:56.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941141d0 con 0x7fdb941041a0 2024-04-20T21:40:56.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.884+0000 7fdb9ac00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941144d0 con 0x7fdb9419a6b0 2024-04-20T21:40:56.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 0x7fdb9419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47770/0 (socket says 172.21.15.7:47770) 2024-04-20T21:40:56.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb99800640 1 -- 172.21.15.7:0/473303225 learned_addr learned my addr 172.21.15.7:0/473303225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:56.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb98e00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb94109270 0x7fdb941a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 -- 172.21.15.7:0/473303225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 msgr2=0x7fdb941a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 -- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb94109270 msgr2=0x7fdb941a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb94109270 0x7fdb941a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb9a200640 1 -- 172.21.15.7:0/473303225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9419b2b0 con 0x7fdb9419a6b0 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.885+0000 7fdb99800640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb9a200640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 0x7fdb9419ab30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009fd0 tx=0x7fdb8c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb92000640 1 -- 172.21.15.7:0/473303225 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00ce20 con 0x7fdb9419a6b0 2024-04-20T21:40:56.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb92000640 1 -- 172.21.15.7:0/473303225 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb8c005020 con 0x7fdb9419a6b0 2024-04-20T21:40:56.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941a7390 con 0x7fdb9419a6b0 2024-04-20T21:40:56.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb92000640 1 -- 172.21.15.7:0/473303225 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c010720 con 0x7fdb9419a6b0 2024-04-20T21:40:56.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.886+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb941a7740 con 0x7fdb9419a6b0 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.889+0000 7fdb9ee6d740 1 Processor -- start 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb92000640 1 -- 172.21.15.7:0/473303225 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdb8c005540 con 0x7fdb9419a6b0 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 -- start start 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5f380 0x5584c5e5f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584c5dc5150 con 0x5584c5e5fca0 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584c5d6b370 con 0x5584c5e68880 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.890+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5584c5dc1110 con 0x5584c5e5f380 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59924/0 (socket says 172.21.15.7:59924) 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 -- 172.21.15.7:0/753194040 learned_addr learned my addr 172.21.15.7:0/753194040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:56.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb98e00640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5f380 0x5584c5e5f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb98e00640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 -- 172.21.15.7:0/753194040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5f380 msgr2=0x5584c5e5f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb92000640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 0x7fdb7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5f380 0x5584c5e5f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 -- 172.21.15.7:0/753194040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 msgr2=0x5584c5e6ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e6ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb99800640 1 -- 172.21.15.7:0/753194040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c5ba17d0 con 0x5584c5e5fca0 2024-04-20T21:40:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb98e00640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e6ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.891+0000 7fdb92000640 1 -- 172.21.15.7:0/473303225 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdb8c0d0da0 con 0x7fdb9419a6b0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.892+0000 7fdb9a200640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 0x7fdb7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.892+0000 7fdb99800640 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb88011650 tx=0x7fdb88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a557d843bf11c4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.892+0000 7fdb82a00640 1 -- 172.21.15.7:0/753194040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8801e070 con 0x5584c5e5fca0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.892+0000 7fdb82a00640 1 -- 172.21.15.7:0/753194040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb8800fba0 con 0x5584c5e5fca0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.892+0000 7fdb82a00640 1 -- 172.21.15.7:0/753194040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88019d40 con 0x5584c5e5fca0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/753194040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 msgr2=0x5584c5e68340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9a200640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 0x7fdb7c0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0991c0 tx=0x7fdb8c005f70 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdb88011650 tx=0x7fdb88009f90 comp rx=0 tx=0).stop 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/753194040 shutdown_connections 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e6ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5fca0 0x5584c5e68340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/753194040 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5f380 0x5584c5e5f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/753194040 >> 172.21.15.7:0/753194040 conn(0x5584c5e59ab0 msgr2=0x5584c5e59e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/753194040 shutdown_connections 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/753194040 wait complete. 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.893+0000 7fdb9ee6d740 1 -- start start 2024-04-20T21:40:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 Processor -- start 2024-04-20T21:40:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584c5e6b670 con 0x5584c5e5f380 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5584c5d7e740 con 0x5584c5e68880 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5584c5e6b7f0 con 0x5584c5e5fca0 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb9ee6d740 10 client.0 _ll_register_callbacks cb 0x5584c5d79b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb98e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb98e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59932/0 (socket says 172.21.15.7:59932) 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.894+0000 7fdb98e00640 1 -- 172.21.15.7:0/3631618695 learned_addr learned my addr 172.21.15.7:0/3631618695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb99800640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 -- 172.21.15.7:0/3631618695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 msgr2=0x5584c5e65eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 -- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 msgr2=0x5584c5e663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 -- 172.21.15.7:0/3631618695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c5ad73f0 con 0x5584c5e5f380 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406b5e0 tx=0x7fdb84009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cfc8d159d44984 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.895+0000 7fdb77400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8402cc50 con 0x5584c5e5f380 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8800fba0 con 0x5584c5e5f380 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdb840046c0 con 0x5584c5e5f380 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb840048c0 con 0x5584c5e5f380 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb9ee6d740 10 client.16425 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584c5a970f0 con 0x5584c5e5f380 2024-04-20T21:40:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb8800fba0 con 0x5584c5e5f380 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.896+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdb84055020 con 0x5584c5e5f380 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 20 client.16425 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5e78c50 0x5584c5e7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 4 client.16425 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584c5dd2b70 con 0x5584c5e78c50 2024-04-20T21:40:56.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.897+0000 7fdb99800640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5e78c50 0x5584c5e7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.898+0000 7fdb99800640 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5e78c50 0x5584c5e7b080 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f77bcc8be6d573 server_cookie=aeeab981acc9a48b in_seq=0 out_seq=0 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.898+0000 7fdb77400640 10 client.16425 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.898+0000 7fdb77400640 1 -- 172.21.15.7:0/3631618695 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584c5dd2b70 con 0x5584c5e78c50 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.898+0000 7fdb77400640 10 client.16425 handle_command_reply: tid=0 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.898+0000 7fdb9ee6d740 1 client.16425 shutdown 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 msgr2=0x5584c5e65970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406b5e0 tx=0x7fdb84009260 comp rx=0 tx=0).stop 2024-04-20T21:40:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 shutdown_connections 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e68880 0x5584c5e663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5e78c50 0x5584c5e7b080 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3631618695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e65970 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 >> 172.21.15.7:0/3631618695 conn(0x5584c5e59ab0 msgr2=0x5584c5e5b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb77400640 0 client.16425 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 shutdown_connections 2024-04-20T21:40:56.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3631618695 wait complete. 2024-04-20T21:40:56.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.899+0000 7fdb9ee6d740 20 client.16425 trim_cache size 0 max 16384 2024-04-20T21:40:56.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 Processor -- start 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 -- start start 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5f380 0x5584c5e74530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 0x5584c5e66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941b2570 con 0x5584c5e68880 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941afe30 con 0x5584c5e5f380 2024-04-20T21:40:56.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.919+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941adca0 con 0x5584c5e5fca0 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46508/0 (socket says 172.21.15.7:46508) 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 -- 172.21.15.7:0/172096177 learned_addr learned my addr 172.21.15.7:0/172096177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb98e00640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5f380 0x5584c5e74530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb98e00640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 0x5584c5e66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 -- 172.21.15.7:0/172096177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5f380 msgr2=0x5584c5e74530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5f380 0x5584c5e74530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 -- 172.21.15.7:0/172096177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 msgr2=0x5584c5e66040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 0x5584c5e66040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 -- 172.21.15.7:0/172096177 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584c5e5fad0 con 0x5584c5e5fca0 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb98e00640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 0x5584c5e66040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.920+0000 7fdb99800640 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb88019850 tx=0x7fdb88019a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70675d3a62979df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb82000640 1 -- 172.21.15.7:0/172096177 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8802f070 con 0x5584c5e5fca0 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb82000640 1 -- 172.21.15.7:0/172096177 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb88029070 con 0x5584c5e5fca0 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb82000640 1 -- 172.21.15.7:0/172096177 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8801e040 con 0x5584c5e5fca0 2024-04-20T21:40:56.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/172096177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 msgr2=0x5584c5e65120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb88019850 tx=0x7fdb88019a40 comp rx=0 tx=0).stop 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/172096177 shutdown_connections 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e68880 0x5584c5e66040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e5fca0 0x5584c5e65120 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/172096177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5f380 0x5584c5e74530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/172096177 >> 172.21.15.7:0/172096177 conn(0x5584c5e59ab0 msgr2=0x5584c5e65930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/172096177 shutdown_connections 2024-04-20T21:40:56.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.921+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/172096177 wait complete. 2024-04-20T21:40:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.922+0000 7fdb9ee6d740 1 -- start start 2024-04-20T21:40:56.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.922+0000 7fdb9ee6d740 1 Processor -- start 2024-04-20T21:40:56.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.922+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.922+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 0x5584c5e6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb9ee6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e68880 0x5584c5e6c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941ae2f0 con 0x5584c5e5f380 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941af760 con 0x5584c5e5fca0 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb9ee6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdb941ae9c0 con 0x5584c5e68880 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb9ee6d740 10 client.0 _ll_register_callbacks cb 0x7fdb84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb98e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 0x5584c5e6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb98e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59940/0 (socket says 172.21.15.7:59940) 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb98e00640 1 -- 172.21.15.7:0/3778299530 learned_addr learned my addr 172.21.15.7:0/3778299530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e68880 0x5584c5e6c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 -- 172.21.15.7:0/3778299530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e68880 msgr2=0x5584c5e6c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e68880 0x5584c5e6c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 -- 172.21.15.7:0/3778299530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 msgr2=0x5584c5e70c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb99800640 1 -- 172.21.15.7:0/3778299530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584c5e6cb80 con 0x5584c5e5fca0 2024-04-20T21:40:56.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.923+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb99800640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 0x5584c5e6c000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5584c5e6b570 tx=0x7fdb88005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a316b053c0cca8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb77400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8802f070 con 0x5584c5e5fca0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb88029070 con 0x5584c5e5fca0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdb88009e40 con 0x5584c5e5fca0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb8801e050 con 0x5584c5e5fca0 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb9ee6d740 10 client.26639 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:56.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.924+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584c5ee7780 con 0x5584c5e5fca0 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb88029070 con 0x5584c5e5fca0 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdb8803a050 con 0x5584c5e5fca0 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 10 client.26639 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 20 client.26639 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5ee7bb0 0x5584c5ee9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:56.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 4 client.26639 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584c5dcc9e0 con 0x5584c5ee7bb0 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.925+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5ee7bb0 0x5584c5ee9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb98e00640 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5ee7bb0 0x5584c5ee9fa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3defe84faca990 server_cookie=ac367f12aac9f0af in_seq=0 out_seq=0 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb77400640 10 client.26639 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb77400640 1 -- 172.21.15.7:0/3778299530 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584c5dcc9e0 con 0x5584c5ee7bb0 2024-04-20T21:40:56.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb77400640 10 client.26639 handle_command_reply: tid=0 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 client.26639 shutdown 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 msgr2=0x5584c5e6c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 0x5584c5e6c000 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x5584c5e6b570 tx=0x7fdb88005770 comp rx=0 tx=0).stop 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 shutdown_connections 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584c5e5fca0 0x5584c5e6c000 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5584c5ee7bb0 0x5584c5ee9fa0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5584c5e68880 0x5584c5e6c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 --2- 172.21.15.7:0/3778299530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584c5e5f380 0x5584c5e70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb77400640 0 client.26639 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.926+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 >> 172.21.15.7:0/3778299530 conn(0x5584c5e59ab0 msgr2=0x5584c5e661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 shutdown_connections 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ee6d740 1 -- 172.21.15.7:0/3778299530 wait complete. 2024-04-20T21:40:56.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ee6d740 20 client.26639 trim_cache size 0 max 16384 2024-04-20T21:40:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:56.936 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:40:56.936 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:40:56.936 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 msgr2=0x7fdb7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 0x7fdb7c0a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0991c0 tx=0x7fdb8c005f70 comp rx=0 tx=0).stop 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 msgr2=0x7fdb9419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.927+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 0x7fdb9419ab30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009fd0 tx=0x7fdb8c00ef10 comp rx=0 tx=0).stop 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 shutdown_connections 2024-04-20T21:40:56.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb9419a6b0 0x7fdb9419ab30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb94109270 0x7fdb941a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdb7c0a3d20 0x7fdb7c0a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 --2- 172.21.15.7:0/473303225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb941041a0 0x7fdb941a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 >> 172.21.15.7:0/473303225 conn(0x7fdb940ffa40 msgr2=0x7fdb94101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 shutdown_connections 2024-04-20T21:40:56.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:56.928+0000 7fdb9ac00640 1 -- 172.21.15.7:0/473303225 wait complete. 2024-04-20T21:40:56.945 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:40:56.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.162+0000 7ff4ed800640 1 -- 172.21.15.7:0/2154843414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8102a60 msgr2=0x7ff4e8106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.162+0000 7ff4ed800640 1 --2- 172.21.15.7:0/2154843414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8102a60 0x7ff4e8106aa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8009ca0 tx=0x7ff4d802c100 comp rx=0 tx=0).stop 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 -- 172.21.15.7:0/2154843414 shutdown_connections 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 --2- 172.21.15.7:0/2154843414 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e8107180 0x7ff4e810ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 --2- 172.21.15.7:0/2154843414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8102a60 0x7ff4e8106aa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 --2- 172.21.15.7:0/2154843414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 -- 172.21.15.7:0/2154843414 >> 172.21.15.7:0/2154843414 conn(0x7ff4e80fd950 msgr2=0x7ff4e80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 -- 172.21.15.7:0/2154843414 shutdown_connections 2024-04-20T21:40:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.163+0000 7ff4ed800640 1 -- 172.21.15.7:0/2154843414 wait complete. 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 Processor -- start 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 -- start start 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8107180 0x7ff4e81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e819a750 0x7ff4e819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e81121f0 con 0x7ff4e8107180 2024-04-20T21:40:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8112070 con 0x7ff4e81020b0 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.164+0000 7ff4ed800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e8112370 con 0x7ff4e819a750 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47848/0 (socket says 172.21.15.7:47848) 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 -- 172.21.15.7:0/3300340631 learned_addr learned my addr 172.21.15.7:0/3300340631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4ece00640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e819a750 0x7ff4e819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7400640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8107180 0x7ff4e81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 -- 172.21.15.7:0/3300340631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e819a750 msgr2=0x7ff4e819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e819a750 0x7ff4e819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 -- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8107180 msgr2=0x7ff4e81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8107180 0x7ff4e81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 -- 172.21.15.7:0/3300340631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e819b350 con 0x7ff4e81020b0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.165+0000 7ff4e7e00640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ea60 tx=0x7ff4d000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.166+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000cde0 con 0x7ff4e81020b0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.166+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4d00035f0 con 0x7ff4e81020b0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.166+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0010670 con 0x7ff4e81020b0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.166+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e81a7430 con 0x7ff4e81020b0 2024-04-20T21:40:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.166+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e81a77e0 con 0x7ff4e81020b0 2024-04-20T21:40:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.167+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e8103580 con 0x7ff4e81020b0 2024-04-20T21:40:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.170+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ff4d00037a0 con 0x7ff4e81020b0 2024-04-20T21:40:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.171+0000 7ff4e4c00640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 0x7ff4cc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.171+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff4d00d0bd0 con 0x7ff4e81020b0 2024-04-20T21:40:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.171+0000 7ff4e7400640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 0x7ff4cc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.172+0000 7ff4e7400640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 0x7ff4cc0a61d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802c710 tx=0x7ff4d8009880 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.176+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ff4d009d230 con 0x7ff4e81020b0 2024-04-20T21:40:57.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.428+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e804fd60 con 0x7ff4e81020b0 2024-04-20T21:40:57.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.429+0000 7ff4e4c00640 1 -- 172.21.15.7:0/3300340631 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7ff4d009cb30 con 0x7ff4e81020b0 2024-04-20T21:40:57.429 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:40:57.430 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:40:57.430 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 msgr2=0x7ff4cc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 0x7ff4cc0a61d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802c710 tx=0x7ff4d8009880 comp rx=0 tx=0).stop 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 msgr2=0x7ff4e81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ea60 tx=0x7ff4d000ef20 comp rx=0 tx=0).stop 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 shutdown_connections 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff4e819a750 0x7ff4e819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e8107180 0x7ff4e81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ff4cc0a3d10 0x7ff4cc0a61d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 --2- 172.21.15.7:0/3300340631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e81020b0 0x7ff4e81a0690 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 >> 172.21.15.7:0/3300340631 conn(0x7ff4e80fd950 msgr2=0x7ff4e80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 shutdown_connections 2024-04-20T21:40:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.432+0000 7ff4ed800640 1 -- 172.21.15.7:0/3300340631 wait complete. 2024-04-20T21:40:57.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:40:57.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.665+0000 7f031ee00640 1 -- 172.21.15.7:0/1593540701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0318109470 msgr2=0x7f0318111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.665+0000 7f031ee00640 1 --2- 172.21.15.7:0/1593540701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0318109470 0x7f0318111040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ca0 tx=0x7f031002c100 comp rx=0 tx=0).stop 2024-04-20T21:40:57.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 -- 172.21.15.7:0/1593540701 shutdown_connections 2024-04-20T21:40:57.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 --2- 172.21.15.7:0/1593540701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0318109470 0x7f0318111040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 --2- 172.21.15.7:0/1593540701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318104d50 0x7f0318108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 --2- 172.21.15.7:0/1593540701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 0x7f0318104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 -- 172.21.15.7:0/1593540701 >> 172.21.15.7:0/1593540701 conn(0x7f03180ffbe0 msgr2=0x7f0318102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 -- 172.21.15.7:0/1593540701 shutdown_connections 2024-04-20T21:40:57.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.666+0000 7f031ee00640 1 -- 172.21.15.7:0/1593540701 wait complete. 2024-04-20T21:40:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.667+0000 7f031ee00640 1 Processor -- start 2024-04-20T21:40:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.667+0000 7f031ee00640 1 -- start start 2024-04-20T21:40:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 0x7f03181a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 0x7f03181a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0318114510 con 0x7f0318109470 2024-04-20T21:40:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0318114390 con 0x7f031819a890 2024-04-20T21:40:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ee00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f0318114690 con 0x7f03181043a0 2024-04-20T21:40:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f0317400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 0x7f03181a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f0317400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 0x7f03181a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59952/0 (socket says 172.21.15.7:59952) 2024-04-20T21:40:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f0317400640 1 -- 172.21.15.7:0/2740988379 learned_addr learned my addr 172.21.15.7:0/2740988379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f0317e00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 0x7f03181a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47862/0 (socket says 172.21.15.7:47862) 2024-04-20T21:40:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 -- 172.21.15.7:0/2740988379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 msgr2=0x7f03181a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 0x7f03181a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 -- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 msgr2=0x7f03181a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 0x7f03181a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.668+0000 7f031ce00640 1 -- 172.21.15.7:0/2740988379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031819b3b0 con 0x7f031819a890 2024-04-20T21:40:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f031ce00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0310009bf0 tx=0x7f0310035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f0314c00640 1 -- 172.21.15.7:0/2740988379 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310039040 con 0x7f031819a890 2024-04-20T21:40:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031819b640 con 0x7f031819a890 2024-04-20T21:40:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f0314c00640 1 -- 172.21.15.7:0/2740988379 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0310044070 con 0x7f031819a890 2024-04-20T21:40:57.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f0314c00640 1 -- 172.21.15.7:0/2740988379 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03100042a0 con 0x7f031819a890 2024-04-20T21:40:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.669+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a7830 con 0x7f031819a890 2024-04-20T21:40:57.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0314c00640 1 -- 172.21.15.7:0/2740988379 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f0310004440 con 0x7f031819a890 2024-04-20T21:40:57.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 Processor -- start 2024-04-20T21:40:57.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 -- start start 2024-04-20T21:40:57.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235af7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556235a5c050 con 0x556235b004b0 2024-04-20T21:40:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556235a02490 con 0x556235af78d0 2024-04-20T21:40:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.673+0000 7f0321bbb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556235a58010 con 0x556235af6fb0 2024-04-20T21:40:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235af7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47866/0 (socket says 172.21.15.7:47866) 2024-04-20T21:40:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235af7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46566/0 (socket says 172.21.15.7:46566) 2024-04-20T21:40:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 -- 172.21.15.7:0/3716677355 learned_addr learned my addr 172.21.15.7:0/3716677355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317400640 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 -- 172.21.15.7:0/3716677355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 msgr2=0x556235af7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235af7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 -- 172.21.15.7:0/3716677355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 msgr2=0x556235b028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 -- 172.21.15.7:0/3716677355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556235a0a070 con 0x556235af78d0 2024-04-20T21:40:57.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.674+0000 7f0317e00640 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0308009c90 tx=0x7f0308011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e849d8776f217973 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f0314c00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 0x7f02fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f030d600640 1 -- 172.21.15.7:0/3716677355 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308005220 con 0x556235af78d0 2024-04-20T21:40:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f030d600640 1 -- 172.21.15.7:0/3716677355 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308004c80 con 0x556235af78d0 2024-04-20T21:40:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f030d600640 1 -- 172.21.15.7:0/3716677355 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308019830 con 0x556235af78d0 2024-04-20T21:40:57.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f0314c00640 1 -- 172.21.15.7:0/2740988379 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f03100f2c10 con 0x7f031819a890 2024-04-20T21:40:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f0317400640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 0x7f02fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f0321bbb740 1 -- 172.21.15.7:0/3716677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 msgr2=0x556235afff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.675+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0308009c90 tx=0x7f0308011810 comp rx=0 tx=0).stop 2024-04-20T21:40:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0317400640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 0x7f02fc0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556235af2870 tx=0x7f0304005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 -- 172.21.15.7:0/3716677355 shutdown_connections 2024-04-20T21:40:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235afff70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3716677355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235af7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 -- 172.21.15.7:0/3716677355 >> 172.21.15.7:0/3716677355 conn(0x556235af16e0 msgr2=0x556235af1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 -- 172.21.15.7:0/3716677355 shutdown_connections 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 -- 172.21.15.7:0/3716677355 wait complete. 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 -- start start 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.676+0000 7f0321bbb740 1 Processor -- start 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235afd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af78d0 0x556235afda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556235b032a0 con 0x556235af78d0 2024-04-20T21:40:57.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556235a15860 con 0x556235b004b0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556235b03420 con 0x556235af6fb0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0321bbb740 10 client.0 _ll_register_callbacks cb 0x5562359e3fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47874/0 (socket says 172.21.15.7:47874) 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 -- 172.21.15.7:0/2164879012 learned_addr learned my addr 172.21.15.7:0/2164879012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0317400640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af78d0 0x556235afda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0317e00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235afd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 -- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 msgr2=0x556235afd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235afd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 -- 172.21.15.7:0/2164879012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af78d0 msgr2=0x556235afda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af78d0 0x556235afda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f031ce00640 1 -- 172.21.15.7:0/2164879012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556235a19950 con 0x556235b004b0 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.677+0000 7f0317e00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235afd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f031ce00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f03100be880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a0d3eeb8f35588 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0302000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03100c3030 con 0x556235b004b0 2024-04-20T21:40:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f03100bb2f0 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0308004c80 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03100c78f0 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0321bbb740 10 client.26657 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.678+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562359e7820 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0308004c80 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f031003c030 con 0x556235b004b0 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 10 client.26657 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 10 client.26657 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 20 client.26657 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b10570 0x556235b129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 4 client.26657 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556235a69a70 con 0x556235b10570 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.679+0000 7f0317e00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b10570 0x556235b129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.680+0000 7f0317e00640 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b10570 0x556235b129f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54ec3391eaabfbe9 server_cookie=4136ea5cf3ce5dba in_seq=0 out_seq=0 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.680+0000 7f0302000640 10 client.26657 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0302000640 1 -- 172.21.15.7:0/2164879012 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556235a69a70 con 0x556235b10570 2024-04-20T21:40:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0302000640 10 client.26657 handle_command_reply: tid=0 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 client.26657 shutdown 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 msgr2=0x556235afdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0310006010 tx=0x7f03100be880 comp rx=0 tx=0).stop 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 shutdown_connections 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b10570 0x556235b129f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235b004b0 0x556235afdfa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af78d0 0x556235afda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2164879012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af6fb0 0x556235afd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 >> 172.21.15.7:0/2164879012 conn(0x556235af16e0 msgr2=0x556235af3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0302000640 0 client.26657 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 shutdown_connections 2024-04-20T21:40:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.681+0000 7f0321bbb740 1 -- 172.21.15.7:0/2164879012 wait complete. 2024-04-20T21:40:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.682+0000 7f0321bbb740 20 client.26657 trim_cache size 0 max 16384 2024-04-20T21:40:57.702 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:57 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3300340631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.696+0000 7f0321bbb740 1 Processor -- start 2024-04-20T21:40:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.696+0000 7f0321bbb740 1 -- start start 2024-04-20T21:40:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235b0c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235b004b0 0x556235b18300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03181b2660 con 0x556235af6fb0 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03181aff20 con 0x556235af78d0 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0321bbb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f03181add90 con 0x556235b004b0 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59972/0 (socket says 172.21.15.7:59972) 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317e00640 1 -- 172.21.15.7:0/2332785617 learned_addr learned my addr 172.21.15.7:0/2332785617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f031ce00640 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235b004b0 0x556235b18300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317400640 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235b0c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317e00640 1 -- 172.21.15.7:0/2332785617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235b004b0 msgr2=0x556235b18300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.697+0000 7f0317e00640 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235b004b0 0x556235b18300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f0317e00640 1 -- 172.21.15.7:0/2332785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 msgr2=0x556235b0c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f0317e00640 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235b0c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f0317e00640 1 -- 172.21.15.7:0/2332785617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55623576e3f0 con 0x556235af6fb0 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f0317e00640 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f030801a000 tx=0x7f0308005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74848b1e81829214 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f030cc00640 1 -- 172.21.15.7:0/2332785617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308033070 con 0x556235af6fb0 2024-04-20T21:40:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f030cc00640 1 -- 172.21.15.7:0/2332785617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030802ddd0 con 0x556235af6fb0 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.698+0000 7f030cc00640 1 -- 172.21.15.7:0/2332785617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308019a40 con 0x556235af6fb0 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- 172.21.15.7:0/2332785617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 msgr2=0x556235affc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f030801a000 tx=0x7f0308005700 comp rx=0 tx=0).stop 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- 172.21.15.7:0/2332785617 shutdown_connections 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af78d0 0x556235b0c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235b004b0 0x556235b18300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 --2- 172.21.15.7:0/2332785617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235af6fb0 0x556235affc50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- 172.21.15.7:0/2332785617 >> 172.21.15.7:0/2332785617 conn(0x556235af16e0 msgr2=0x556235af88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- 172.21.15.7:0/2332785617 shutdown_connections 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- 172.21.15.7:0/2332785617 wait complete. 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.699+0000 7f0321bbb740 1 -- start start 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 Processor -- start 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af78d0 0x556235b03cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03181ae3e0 con 0x556235b004b0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03181af850 con 0x556235af6fb0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f03181aeab0 con 0x556235af78d0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0321bbb740 10 client.0 _ll_register_callbacks cb 0x7f03100bea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0317e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47888/0 (socket says 172.21.15.7:47888) 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f0317e00640 1 -- 172.21.15.7:0/3078559196 learned_addr learned my addr 172.21.15.7:0/3078559196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.700+0000 7f031ce00640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317400640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af78d0 0x556235b03cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 -- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af78d0 msgr2=0x556235b03cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af78d0 0x556235b03cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 -- 172.21.15.7:0/3078559196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 msgr2=0x556235b041f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b041f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 -- 172.21.15.7:0/3078559196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562358387d0 con 0x556235af6fb0 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0317e00640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0308019660 tx=0x7f0308009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e500792b703d0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0302000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308033070 con 0x556235af6fb0 2024-04-20T21:40:57.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f03080050d0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.701+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f030802ddd0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03080052d0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 10 client.26663 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556235b768a0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030802ddd0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f03080035f0 con 0x556235af6fb0 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 10 client.26663 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 10 client.26663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 20 client.26663 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b76d60 0x556235b79150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 4 client.26663 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.702+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556235a638e0 con 0x556235b76d60 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.703+0000 7f0317400640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b76d60 0x556235b79150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.703+0000 7f0317400640 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b76d60 0x556235b79150 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0688f89dfb09291 server_cookie=d7d86c6e240f5156 in_seq=0 out_seq=0 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.703+0000 7f0302000640 10 client.26663 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0302000640 1 -- 172.21.15.7:0/3078559196 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556235a638e0 con 0x556235b76d60 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0302000640 10 client.26663 handle_command_reply: tid=0 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 client.26663 shutdown 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 msgr2=0x556235b08940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0308019660 tx=0x7f0308009c50 comp rx=0 tx=0).stop 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 shutdown_connections 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556235b004b0 0x556235b041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556235af78d0 0x556235b03cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x556235b76d60 0x556235b79150 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 --2- 172.21.15.7:0/3078559196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556235af6fb0 0x556235b08940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 >> 172.21.15.7:0/3078559196 conn(0x556235af16e0 msgr2=0x556235afd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0302000640 0 client.26663 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 shutdown_connections 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 1 -- 172.21.15.7:0/3078559196 wait complete. 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.704+0000 7f0321bbb740 20 client.26663 trim_cache size 0 max 16384 2024-04-20T21:40:57.717 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 msgr2=0x7f02fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 0x7f02fc0a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x556235af2870 tx=0x7f0304005f90 comp rx=0 tx=0).stop 2024-04-20T21:40:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 msgr2=0x7f031819ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0310009bf0 tx=0x7f0310035d10 comp rx=0 tx=0).stop 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 shutdown_connections 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f02fc0a3d20 0x7f02fc0a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f031819a890 0x7f031819ad40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0318109470 0x7f03181a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 --2- 172.21.15.7:0/2740988379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f03181043a0 0x7f03181a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 >> 172.21.15.7:0/2740988379 conn(0x7f03180ffbe0 msgr2=0x7f03181007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 shutdown_connections 2024-04-20T21:40:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.705+0000 7f031ee00640 1 -- 172.21.15.7:0/2740988379 wait complete. 2024-04-20T21:40:57.720 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:57 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3300340631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:57.721 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:57 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3300340631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:40:57.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.943+0000 7f924b000640 1 -- 172.21.15.7:0/2570156643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924410e3e0 msgr2=0x7f924410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.943+0000 7f924b000640 1 --2- 172.21.15.7:0/2570156643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924410e3e0 0x7f924410e7c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9238009890 tx=0x7f923802bb70 comp rx=0 tx=0).stop 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 -- 172.21.15.7:0/2570156643 shutdown_connections 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 --2- 172.21.15.7:0/2570156643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9244108e40 0x7f9244115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 --2- 172.21.15.7:0/2570156643 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f92441084a0 0x7f9244108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 --2- 172.21.15.7:0/2570156643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f924410e3e0 0x7f924410e7c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 -- 172.21.15.7:0/2570156643 >> 172.21.15.7:0/2570156643 conn(0x7f9244077d70 msgr2=0x7f9244106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 -- 172.21.15.7:0/2570156643 shutdown_connections 2024-04-20T21:40:57.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.944+0000 7f924b000640 1 -- 172.21.15.7:0/2570156643 wait complete. 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 Processor -- start 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 -- start start 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9244108e40 0x7f92441a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f924410e3e0 0x7f924419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92441186f0 con 0x7f92441084a0 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9244118570 con 0x7f924410e3e0 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.945+0000 7f924b000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9244118870 con 0x7f9244108e40 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9244108e40 0x7f92441a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60006/0 (socket says 172.21.15.7:60006) 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 -- 172.21.15.7:0/1175114309 learned_addr learned my addr 172.21.15.7:0/1175114309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f924a600640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f924410e3e0 0x7f924419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 -- 172.21.15.7:0/1175114309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9244108e40 msgr2=0x7f92441a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9244108e40 0x7f92441a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 -- 172.21.15.7:0/1175114309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f924410e3e0 msgr2=0x7f924419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f924410e3e0 0x7f924419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.946+0000 7f9249c00640 1 -- 172.21.15.7:0/1175114309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924419f270 con 0x7f92441084a0 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f9249c00640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f92380042f0 tx=0x7f9238004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f9242000640 1 -- 172.21.15.7:0/1175114309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238041020 con 0x7f92441084a0 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f9242000640 1 -- 172.21.15.7:0/1175114309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923802c9f0 con 0x7f92441084a0 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924419f500 con 0x7f92441084a0 2024-04-20T21:40:57.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f9242000640 1 -- 172.21.15.7:0/1175114309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238039040 con 0x7f92441084a0 2024-04-20T21:40:57.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.947+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441ab800 con 0x7f92441084a0 2024-04-20T21:40:57.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.950+0000 7f924f3b7740 1 Processor -- start 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 -- start start 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b849130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b84ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56029b7a5050 con 0x56029b840170 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56029b74b490 con 0x56029b840a90 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924f3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56029b7a1010 con 0x56029b849670 2024-04-20T21:40:57.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f9242000640 1 -- 172.21.15.7:0/1175114309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9238044020 con 0x7f92441084a0 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.951+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60016/0 (socket says 172.21.15.7:60016) 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 -- 172.21.15.7:0/3198473344 learned_addr learned my addr 172.21.15.7:0/3198473344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f9249200640 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b84ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f9249c00640 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b849130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 -- 172.21.15.7:0/3198473344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 msgr2=0x56029b84ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b84ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 -- 172.21.15.7:0/3198473344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 msgr2=0x56029b849130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b849130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 -- 172.21.15.7:0/3198473344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029b753070 con 0x56029b840170 2024-04-20T21:40:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.952+0000 7f924a600640 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f923c009f50 tx=0x7f923c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83845498bbc747e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9236a00640 1 -- 172.21.15.7:0/3198473344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c004d20 con 0x56029b840170 2024-04-20T21:40:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9236a00640 1 -- 172.21.15.7:0/3198473344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c004050 con 0x56029b840170 2024-04-20T21:40:57.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9236a00640 1 -- 172.21.15.7:0/3198473344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c005280 con 0x56029b840170 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9242000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 0x7f922c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9242000640 1 -- 172.21.15.7:0/1175114309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f92380f1fe0 con 0x7f92441084a0 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f9249c00640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 0x7f922c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.953+0000 7f924f3b7740 1 -- 172.21.15.7:0/3198473344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 msgr2=0x56029b840550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f923c009f50 tx=0x7f923c0118d0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 -- 172.21.15.7:0/3198473344 shutdown_connections 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b849130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b84ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3198473344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b840550 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 -- 172.21.15.7:0/3198473344 >> 172.21.15.7:0/3198473344 conn(0x56029b83a8a0 msgr2=0x56029b83ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 -- 172.21.15.7:0/3198473344 shutdown_connections 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 -- 172.21.15.7:0/3198473344 wait complete. 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f9249c00640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 0x7f922c0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x56029b83fc40 tx=0x7f92380f5ad0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.954+0000 7f924f3b7740 1 -- start start 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 Processor -- start 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b840170 0x56029b8466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 0x56029b847150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56029b84c460 con 0x56029b849670 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56029b75e860 con 0x56029b840a90 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56029b84c5e0 con 0x56029b840170 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.955+0000 7f924f3b7740 10 client.0 _ll_register_callbacks cb 0x56029b72cfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 0x56029b847150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 0x56029b847150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60026/0 (socket says 172.21.15.7:60026) 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f924a600640 1 -- 172.21.15.7:0/619063379 learned_addr learned my addr 172.21.15.7:0/619063379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f924a600640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b840170 0x56029b8466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 -- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b840170 msgr2=0x56029b8466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b840170 0x56029b8466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 -- 172.21.15.7:0/619063379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 msgr2=0x56029b847150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 0x56029b847150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 -- 172.21.15.7:0/619063379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029b762950 con 0x56029b840a90 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f9249200640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f924419fc30 tx=0x7f9230011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ccae505e616516 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.956+0000 7f922b400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923001e020 con 0x56029b840a90 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f923c004050 con 0x56029b840a90 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f923000fbc0 con 0x56029b840a90 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f923000fdc0 con 0x56029b840a90 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f924f3b7740 10 client.26675 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:57.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.957+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56029b730820 con 0x56029b840a90 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c004050 con 0x56029b840a90 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9230019070 con 0x56029b840a90 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 10 client.26675 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 10 client.26675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 20 client.26675 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b859760 0x56029b85bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 4 client.26675 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:57.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.958+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56029b7b2a70 con 0x56029b859760 2024-04-20T21:40:57.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.959+0000 7f924a600640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b859760 0x56029b85bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.959+0000 7f924a600640 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b859760 0x56029b85bb80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d8b6b02a21140e4 server_cookie=a119707315e80cae in_seq=0 out_seq=0 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.959+0000 7f922b400640 10 client.26675 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f922b400640 1 -- 172.21.15.7:0/619063379 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56029b7b2a70 con 0x56029b859760 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f922b400640 10 client.26675 handle_command_reply: tid=0 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 client.26675 shutdown 2024-04-20T21:40:57.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 msgr2=0x56029b846c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f924419fc30 tx=0x7f9230011b50 comp rx=0 tx=0).stop 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 shutdown_connections 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b859760 0x56029b85bb80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b849670 0x56029b847150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 --2- 172.21.15.7:0/619063379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b840170 0x56029b8466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 >> 172.21.15.7:0/619063379 conn(0x56029b83a8a0 msgr2=0x56029b83c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f922b400640 0 client.26675 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 shutdown_connections 2024-04-20T21:40:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.960+0000 7f924f3b7740 1 -- 172.21.15.7:0/619063379 wait complete. 2024-04-20T21:40:57.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.961+0000 7f924f3b7740 20 client.26675 trim_cache size 0 max 16384 2024-04-20T21:40:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.975+0000 7f924f3b7740 1 Processor -- start 2024-04-20T21:40:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.975+0000 7f924f3b7740 1 -- start start 2024-04-20T21:40:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b861450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b69d0 con 0x56029b840170 2024-04-20T21:40:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b4290 con 0x56029b840a90 2024-04-20T21:40:57.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924f3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b2100 con 0x56029b849670 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60028/0 (socket says 172.21.15.7:60028) 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924a600640 1 -- 172.21.15.7:0/794496925 learned_addr learned my addr 172.21.15.7:0/794496925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f9249200640 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.976+0000 7f924a600640 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b861450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 -- 172.21.15.7:0/794496925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 msgr2=0x56029b861450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b861450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 -- 172.21.15.7:0/794496925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 msgr2=0x56029b846da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 -- 172.21.15.7:0/794496925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029b4b73f0 con 0x56029b840170 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f924a600640 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f923c012fd0 tx=0x7f923c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ac83cf6b539d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f9236000640 1 -- 172.21.15.7:0/794496925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c02fe60 con 0x56029b840170 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f9236000640 1 -- 172.21.15.7:0/794496925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c035070 con 0x56029b840170 2024-04-20T21:40:57.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.977+0000 7f9236000640 1 -- 172.21.15.7:0/794496925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923c0279a0 con 0x56029b840170 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- 172.21.15.7:0/794496925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 msgr2=0x56029b855350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f923c012fd0 tx=0x7f923c0047b0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- 172.21.15.7:0/794496925 shutdown_connections 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840a90 0x56029b846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b861450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 --2- 172.21.15.7:0/794496925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840170 0x56029b855350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- 172.21.15.7:0/794496925 >> 172.21.15.7:0/794496925 conn(0x56029b83a8a0 msgr2=0x56029b846690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- 172.21.15.7:0/794496925 shutdown_connections 2024-04-20T21:40:57.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- 172.21.15.7:0/794496925 wait complete. 2024-04-20T21:40:57.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.978+0000 7f924f3b7740 1 -- start start 2024-04-20T21:40:57.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 Processor -- start 2024-04-20T21:40:57.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 0x56029b84d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b8becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b2750 con 0x56029b840a90 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b3bc0 con 0x56029b840170 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f92441b2e20 con 0x56029b849670 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f924f3b7740 10 client.0 _ll_register_callbacks cb 0x7f9230009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.979+0000 7f9249200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f924a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 0x56029b84d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60030/0 (socket says 172.21.15.7:60030) 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f924a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 0x56029b84d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47968/0 (socket says 172.21.15.7:47968) 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 -- 172.21.15.7:0/3620506268 learned_addr learned my addr 172.21.15.7:0/3620506268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f924a600640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b8becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 -- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 msgr2=0x56029b8becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b8becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 -- 172.21.15.7:0/3620506268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 msgr2=0x56029b84d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:57.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 0x56029b84d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 -- 172.21.15.7:0/3620506268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029b5817d0 con 0x56029b840a90 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f9249200640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56029b847ca0 tx=0x7f9230022610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7689b9b33dd84a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.980+0000 7f922b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230031060 con 0x56029b840a90 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f923000fbc0 con 0x56029b840a90 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f923c035070 con 0x56029b840a90 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f923001e040 con 0x56029b840a90 2024-04-20T21:40:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f924f3b7740 10 client.16470 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56029b8c01c0 con 0x56029b840a90 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923c035070 con 0x56029b840a90 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9230019070 con 0x56029b840a90 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.981+0000 7f924f3b7740 10 client.16470 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924f3b7740 10 client.16470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924f3b7740 20 client.16470 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b8c0970 0x56029b8c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924f3b7740 4 client.16470 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56029b7ac8e0 con 0x56029b8c0970 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924a600640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b8c0970 0x56029b8c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f924a600640 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b8c0970 0x56029b8c2d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86711ca4be5cca7f server_cookie=b6eb43db43bdb8c5 in_seq=0 out_seq=0 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.982+0000 7f922b400640 10 client.16470 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f922b400640 1 -- 172.21.15.7:0/3620506268 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56029b7ac8e0 con 0x56029b8c0970 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f922b400640 10 client.16470 handle_command_reply: tid=0 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 client.16470 shutdown 2024-04-20T21:40:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 msgr2=0x56029b84db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x56029b847ca0 tx=0x7f9230022610 comp rx=0 tx=0).stop 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 shutdown_connections 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56029b8c0970 0x56029b8c2d90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56029b849670 0x56029b8becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56029b840a90 0x56029b84db70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 --2- 172.21.15.7:0/3620506268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56029b840170 0x56029b84d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f922b400640 0 client.16470 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:40:57.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 >> 172.21.15.7:0/3620506268 conn(0x56029b83a8a0 msgr2=0x56029b8390d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 shutdown_connections 2024-04-20T21:40:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 1 -- 172.21.15.7:0/3620506268 wait complete. 2024-04-20T21:40:57.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.983+0000 7f924f3b7740 20 client.16470 trim_cache size 0 max 16384 2024-04-20T21:40:57.991 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 msgr2=0x7f922c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 0x7f922c0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x56029b83fc40 tx=0x7f92380f5ad0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 msgr2=0x7f92441a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:57.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f92380042f0 tx=0x7f9238004320 comp rx=0 tx=0).stop 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 shutdown_connections 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f922c0a3d20 0x7f922c0a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f924410e3e0 0x7f924419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9244108e40 0x7f92441a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 --2- 172.21.15.7:0/1175114309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92441084a0 0x7f92441a4940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 >> 172.21.15.7:0/1175114309 conn(0x7f9244077d70 msgr2=0x7f9244078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 shutdown_connections 2024-04-20T21:40:57.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:57.984+0000 7f924b000640 1 -- 172.21.15.7:0/1175114309 wait complete. 2024-04-20T21:40:58.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:40:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.207+0000 7f6a5fe00640 1 -- 172.21.15.7:0/1402273871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 msgr2=0x7f6a60103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.207+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/1402273871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a60103320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009ca0 tx=0x7f6a5002c190 comp rx=0 tx=0).stop 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 -- 172.21.15.7:0/1402273871 shutdown_connections 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/1402273871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a6010c360 0x7f6a6010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/1402273871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a60103860 0x7f6a6010bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/1402273871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a60103320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 -- 172.21.15.7:0/1402273871 >> 172.21.15.7:0/1402273871 conn(0x7f6a60077a10 msgr2=0x7f6a600fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 -- 172.21.15.7:0/1402273871 shutdown_connections 2024-04-20T21:40:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.208+0000 7f6a5fe00640 1 -- 172.21.15.7:0/1402273871 wait complete. 2024-04-20T21:40:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.209+0000 7f6a5fe00640 1 Processor -- start 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.209+0000 7f6a5fe00640 1 -- start start 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60103860 0x7f6a601a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6010c360 0x7f6a6019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112110 con 0x7f6a6010c360 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60111f90 con 0x7f6a60102f40 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a60112290 con 0x7f6a60103860 2024-04-20T21:40:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47978/0 (socket says 172.21.15.7:47978) 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5ea00640 1 -- 172.21.15.7:0/43521307 learned_addr learned my addr 172.21.15.7:0/43521307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5f400640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6010c360 0x7f6a6019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.210+0000 7f6a5e000640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60103860 0x7f6a601a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 -- 172.21.15.7:0/43521307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60103860 msgr2=0x7f6a601a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60103860 0x7f6a601a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 -- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6010c360 msgr2=0x7f6a6019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6010c360 0x7f6a6019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 -- 172.21.15.7:0/43521307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6019af40 con 0x7f6a60102f40 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5ea00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005e30 tx=0x7f6a50035d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a57400640 1 -- 172.21.15.7:0/43521307 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50039040 con 0x7f6a60102f40 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.211+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6019b1d0 con 0x7f6a60102f40 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.212+0000 7f6a57400640 1 -- 172.21.15.7:0/43521307 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50044070 con 0x7f6a60102f40 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.212+0000 7f6a57400640 1 -- 172.21.15.7:0/43521307 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a500328d0 con 0x7f6a60102f40 2024-04-20T21:40:58.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.212+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a7540 con 0x7f6a60102f40 2024-04-20T21:40:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 Processor -- start 2024-04-20T21:40:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a57400640 1 -- 172.21.15.7:0/43521307 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6a50032a70 con 0x7f6a60102f40 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 -- start start 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec32670 0x56277ec34a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56277eb8e050 con 0x56277ec29a90 2024-04-20T21:40:58.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56277eb34490 con 0x56277ec29170 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.215+0000 7f6a6863c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56277eb8a010 con 0x56277ec32670 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:47982/0 (socket says 172.21.15.7:47982) 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 -- 172.21.15.7:0/3126018400 learned_addr learned my addr 172.21.15.7:0/3126018400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5e000640 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec32670 0x56277ec34a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 -- 172.21.15.7:0/3126018400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec32670 msgr2=0x56277ec34a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec32670 0x56277ec34a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 -- 172.21.15.7:0/3126018400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 msgr2=0x56277ec32130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec32130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.216+0000 7f6a5f400640 1 -- 172.21.15.7:0/3126018400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277eb3c070 con 0x56277ec29170 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009c20 tx=0x7f6a58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6eabd39deb1d786 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a4fe00640 1 -- 172.21.15.7:0/3126018400 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5801e070 con 0x56277ec29170 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a4fe00640 1 -- 172.21.15.7:0/3126018400 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a58019b10 con 0x56277ec29170 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a4fe00640 1 -- 172.21.15.7:0/3126018400 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800eae0 con 0x56277ec29170 2024-04-20T21:40:58.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a57400640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 0x7f6a440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a57400640 1 -- 172.21.15.7:0/43521307 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f6a500f28b0 con 0x7f6a60102f40 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a5ea00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 0x7f6a440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a6863c740 1 -- 172.21.15.7:0/3126018400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 msgr2=0x56277ec29550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.217+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009c20 tx=0x7f6a58011810 comp rx=0 tx=0).stop 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 -- 172.21.15.7:0/3126018400 shutdown_connections 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec32670 0x56277ec34a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec32130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3126018400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec29550 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 -- 172.21.15.7:0/3126018400 >> 172.21.15.7:0/3126018400 conn(0x56277ec238a0 msgr2=0x56277ec23c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 -- 172.21.15.7:0/3126018400 shutdown_connections 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 -- 172.21.15.7:0/3126018400 wait complete. 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a5ea00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 0x7f6a440a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56277ec28c40 tx=0x7f6a500f8530 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.218+0000 7f6a6863c740 1 -- start start 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 Processor -- start 2024-04-20T21:40:58.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 0x56277ec2fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56277ec35460 con 0x56277ec32670 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56277eb47860 con 0x56277ec29a90 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56277ec355e0 con 0x56277ec29170 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a6863c740 10 client.0 _ll_register_callbacks cb 0x56277eb15fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.219+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60056/0 (socket says 172.21.15.7:60056) 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5f400640 1 -- 172.21.15.7:0/2116393303 learned_addr learned my addr 172.21.15.7:0/2116393303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 0x56277ec2fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 -- 172.21.15.7:0/2116393303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 msgr2=0x56277ec2f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 -- 172.21.15.7:0/2116393303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 msgr2=0x56277ec30140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 -- 172.21.15.7:0/2116393303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277eb4b950 con 0x56277ec29a90 2024-04-20T21:40:58.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a5e000640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 0x56277ec2fc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a6019bed0 tx=0x7f6a48012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e16081a78a8f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.220+0000 7f6a4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800f920 con 0x56277ec29a90 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a58019b10 con 0x56277ec29a90 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f6a4800fac0 con 0x56277ec29a90 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a4800fcc0 con 0x56277ec29a90 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a6863c740 10 client.26693 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:58.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56277eb19820 con 0x56277ec29a90 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.221+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a58019b10 con 0x56277ec29a90 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f6a48023070 con 0x56277ec29a90 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 10 client.26693 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 20 client.26693 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277ec42780 0x56277ec44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 4 client.26693 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:58.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56277eb9ba70 con 0x56277ec42780 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.222+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277ec42780 0x56277ec44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.223+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277ec42780 0x56277ec44ba0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab91416cec79ea7 server_cookie=5193f6304bc155cc in_seq=0 out_seq=0 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.223+0000 7f6a4cc00640 10 client.26693 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a4cc00640 1 -- 172.21.15.7:0/2116393303 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56277eb9ba70 con 0x56277ec42780 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a4cc00640 10 client.26693 handle_command_reply: tid=0 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 client.26693 shutdown 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 msgr2=0x56277ec2fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 0x56277ec2fc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a6019bed0 tx=0x7f6a48012b60 comp rx=0 tx=0).stop 2024-04-20T21:40:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 shutdown_connections 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29a90 0x56277ec2fc00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277ec42780 0x56277ec44ba0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec30140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2116393303 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 >> 172.21.15.7:0/2116393303 conn(0x56277ec238a0 msgr2=0x56277ec255f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a4cc00640 0 client.26693 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.224+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 shutdown_connections 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.225+0000 7f6a6863c740 1 -- 172.21.15.7:0/2116393303 wait complete. 2024-04-20T21:40:58.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.225+0000 7f6a6863c740 20 client.26693 trim_cache size 0 max 16384 2024-04-20T21:40:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.239+0000 7f6a6863c740 1 Processor -- start 2024-04-20T21:40:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.239+0000 7f6a6863c740 1 -- start start 2024-04-20T21:40:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec31df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29a90 0x56277ec3e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601b2510 con 0x56277ec32670 2024-04-20T21:40:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601afdd0 con 0x56277ec29170 2024-04-20T21:40:58.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a6863c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601adc40 con 0x56277ec29a90 2024-04-20T21:40:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec31df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec31df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48002/0 (socket says 172.21.15.7:48002) 2024-04-20T21:40:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a5f400640 1 -- 172.21.15.7:0/2203857404 learned_addr learned my addr 172.21.15.7:0/2203857404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:58.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.240+0000 7f6a5e000640 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29a90 0x56277ec3e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 -- 172.21.15.7:0/2203857404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29a90 msgr2=0x56277ec3e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29a90 0x56277ec3e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 -- 172.21.15.7:0/2203857404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 msgr2=0x56277ec31df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec31df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 -- 172.21.15.7:0/2203857404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56277e8a03f0 con 0x56277ec32670 2024-04-20T21:40:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a5f400640 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec2fd90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005940 tx=0x7f6a5800e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe4ab37d3b9f41c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a4f400640 1 -- 172.21.15.7:0/2203857404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5801e040 con 0x56277ec32670 2024-04-20T21:40:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a4f400640 1 -- 172.21.15.7:0/2203857404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5800e770 con 0x56277ec32670 2024-04-20T21:40:58.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.241+0000 7f6a4f400640 1 -- 172.21.15.7:0/2203857404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58019b20 con 0x56277ec32670 2024-04-20T21:40:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- 172.21.15.7:0/2203857404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 msgr2=0x56277ec2fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec2fd90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6a58005940 tx=0x7f6a5800e640 comp rx=0 tx=0).stop 2024-04-20T21:40:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- 172.21.15.7:0/2203857404 shutdown_connections 2024-04-20T21:40:58.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec32670 0x56277ec2fd90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29a90 0x56277ec3e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 --2- 172.21.15.7:0/2203857404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec29170 0x56277ec31df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- 172.21.15.7:0/2203857404 >> 172.21.15.7:0/2203857404 conn(0x56277ec238a0 msgr2=0x56277ec2f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- 172.21.15.7:0/2203857404 shutdown_connections 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- 172.21.15.7:0/2203857404 wait complete. 2024-04-20T21:40:58.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.242+0000 7f6a6863c740 1 -- start start 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 Processor -- start 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 0x56277ec2cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601ae290 con 0x56277ec29a90 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601af700 con 0x56277ec32670 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a601ae960 con 0x56277ec29170 2024-04-20T21:40:58.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a6863c740 10 client.0 _ll_register_callbacks cb 0x7f6a48009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a5e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a5e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60074/0 (socket says 172.21.15.7:60074) 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.243+0000 7f6a5e000640 1 -- 172.21.15.7:0/3241993713 learned_addr learned my addr 172.21.15.7:0/3241993713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 0x56277ec2cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 -- 172.21.15.7:0/3241993713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 msgr2=0x56277ec2c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 -- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 msgr2=0x56277ec2cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 0x56277ec2cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 -- 172.21.15.7:0/3241993713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56277e96a7d0 con 0x56277ec29a90 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 0x56277ec2cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a5e000640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009db0 tx=0x7f6a48005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49090353394fc081 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48029070 con 0x56277ec29a90 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f6a48005680 con 0x56277ec29a90 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.244+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5800e770 con 0x56277ec29a90 2024-04-20T21:40:58.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a4800fe60 con 0x56277ec29a90 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 10 client.16488 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56277eca8f20 con 0x56277ec29a90 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5800e770 con 0x56277ec29a90 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f6a48034020 con 0x56277ec29a90 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 10 client.16488 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 20 client.16488 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:58.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277eca9680 0x56277ecabaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 4 client.16488 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.245+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56277eb958e0 con 0x56277eca9680 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277eca9680 0x56277ecabaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a5f400640 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277eca9680 0x56277ecabaa0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddbe94e99f4e3d96 server_cookie=7b9327248592c50d in_seq=0 out_seq=0 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a4cc00640 10 client.16488 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a4cc00640 1 -- 172.21.15.7:0/3241993713 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56277eb958e0 con 0x56277eca9680 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a4cc00640 10 client.16488 handle_command_reply: tid=0 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.246+0000 7f6a6863c740 1 client.16488 shutdown 2024-04-20T21:40:58.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 msgr2=0x56277ec2c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009db0 tx=0x7f6a48005280 comp rx=0 tx=0).stop 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 shutdown_connections 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56277ec32670 0x56277ec2cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56277eca9680 0x56277ecabaa0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56277ec29a90 0x56277ec2c9c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 --2- 172.21.15.7:0/3241993713 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56277ec29170 0x56277ec2c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 >> 172.21.15.7:0/3241993713 conn(0x56277ec238a0 msgr2=0x56277ec254d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a4cc00640 0 client.16488 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 shutdown_connections 2024-04-20T21:40:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 1 -- 172.21.15.7:0/3241993713 wait complete. 2024-04-20T21:40:58.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.247+0000 7f6a6863c740 20 client.16488 trim_cache size 0 max 16384 2024-04-20T21:40:58.259 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:40:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 msgr2=0x7f6a440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 0x7f6a440a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56277ec28c40 tx=0x7f6a500f8530 comp rx=0 tx=0).stop 2024-04-20T21:40:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 msgr2=0x7f6a601a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:58.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a50005e30 tx=0x7f6a50035d90 comp rx=0 tx=0).stop 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 shutdown_connections 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6010c360 0x7f6a6019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60103860 0x7f6a601a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f6a440a3da0 0x7f6a440a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 --2- 172.21.15.7:0/43521307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a60102f40 0x7f6a601a05c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 >> 172.21.15.7:0/43521307 conn(0x7f6a60077a10 msgr2=0x7f6a600fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 shutdown_connections 2024-04-20T21:40:58.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:58.248+0000 7f6a5fe00640 1 -- 172.21.15.7:0/43521307 wait complete. 2024-04-20T21:40:58.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:40:58 smithi007 ceph-mon[16867]: pgmap v234: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:40:58 smithi016 ceph-mon[21409]: pgmap v234: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:40:58 smithi186 ceph-mon[22157]: pgmap v234: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:40:59.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:40:59.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:40:59.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:40:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 -- 172.21.15.7:0/476862456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c104b70 msgr2=0x7f2a5c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 --2- 172.21.15.7:0/476862456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c104b70 0x7f2a5c108bb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009900 tx=0x7f2a4402bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 -- 172.21.15.7:0/476862456 shutdown_connections 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 --2- 172.21.15.7:0/476862456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c104b70 0x7f2a5c108bb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 --2- 172.21.15.7:0/476862456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c109290 0x7f2a5c110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 --2- 172.21.15.7:0/476862456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 0x7f2a5c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.501+0000 7f2a5b200640 1 -- 172.21.15.7:0/476862456 >> 172.21.15.7:0/476862456 conn(0x7f2a5c0ffa00 msgr2=0x7f2a5c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.502+0000 7f2a5b200640 1 -- 172.21.15.7:0/476862456 shutdown_connections 2024-04-20T21:40:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.502+0000 7f2a5b200640 1 -- 172.21.15.7:0/476862456 wait complete. 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 Processor -- start 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 -- start start 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 0x7f2a5c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c109290 0x7f2a5c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c114320 con 0x7f2a5c19a740 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1141a0 con 0x7f2a5c109290 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.503+0000 7f2a5b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1144a0 con 0x7f2a5c1041c0 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60098/0 (socket says 172.21.15.7:60098) 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 -- 172.21.15.7:0/240730080 learned_addr learned my addr 172.21.15.7:0/240730080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a53e00640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c109290 0x7f2a5c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a59e00640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 0x7f2a5c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 -- 172.21.15.7:0/240730080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 msgr2=0x7f2a5c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 0x7f2a5c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 -- 172.21.15.7:0/240730080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c109290 msgr2=0x7f2a5c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c109290 0x7f2a5c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 -- 172.21.15.7:0/240730080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c19b340 con 0x7f2a5c19a740 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.504+0000 7f2a5a800640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400efd0 tx=0x7f2a5400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.505+0000 7f2a52a00640 1 -- 172.21.15.7:0/240730080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54003b70 con 0x7f2a5c19a740 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.505+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c1a73d0 con 0x7f2a5c19a740 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.505+0000 7f2a52a00640 1 -- 172.21.15.7:0/240730080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a54003d10 con 0x7f2a5c19a740 2024-04-20T21:40:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.505+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a7780 con 0x7f2a5c19a740 2024-04-20T21:40:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.505+0000 7f2a52a00640 1 -- 172.21.15.7:0/240730080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54010990 con 0x7f2a5c19a740 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.508+0000 7f2a639a4740 1 Processor -- start 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.508+0000 7f2a52a00640 1 -- 172.21.15.7:0/240730080 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2a54004000 con 0x7f2a5c19a740 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 -- start start 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e5185e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e518670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e518699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e517c2d20 con 0x560e5185e0f0 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e517693e0 con 0x560e5185ea10 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a639a4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560e517bece0 con 0x560e518675f0 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e5185e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e5185e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60102/0 (socket says 172.21.15.7:60102) 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a53e00640 1 -- 172.21.15.7:0/4053578971 learned_addr learned my addr 172.21.15.7:0/4053578971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.509+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e518670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a53e00640 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e518699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 -- 172.21.15.7:0/4053578971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 msgr2=0x560e518699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e518699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 -- 172.21.15.7:0/4053578971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 msgr2=0x560e5185e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e5185e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 -- 172.21.15.7:0/4053578971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e51788980 con 0x560e5185ea10 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a52a00640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 0x7f2a400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e518670b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009c00 tx=0x7f2a4c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1adca3a8ad1982 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a52a00640 1 -- 172.21.15.7:0/240730080 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f2a540d0f80 con 0x7f2a5c19a740 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a4b000640 1 -- 172.21.15.7:0/4053578971 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004230 con 0x560e5185ea10 2024-04-20T21:40:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.510+0000 7f2a53e00640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 0x7f2a400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a4b000640 1 -- 172.21.15.7:0/4053578971 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c0043d0 con 0x560e5185ea10 2024-04-20T21:40:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a4b000640 1 -- 172.21.15.7:0/4053578971 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004d90 con 0x560e5185ea10 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 -- 172.21.15.7:0/4053578971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 msgr2=0x560e518670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e518670b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009c00 tx=0x7f2a4c011810 comp rx=0 tx=0).stop 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a53e00640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 0x7f2a400a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560e518599b0 tx=0x7f2a44005dd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 -- 172.21.15.7:0/4053578971 shutdown_connections 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e518670b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e518699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4053578971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e5185e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 -- 172.21.15.7:0/4053578971 >> 172.21.15.7:0/4053578971 conn(0x560e51858820 msgr2=0x560e51858c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 -- 172.21.15.7:0/4053578971 shutdown_connections 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.511+0000 7f2a639a4740 1 -- 172.21.15.7:0/4053578971 wait complete. 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.512+0000 7f2a639a4740 1 -- start start 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.512+0000 7f2a639a4740 1 Processor -- start 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e51864aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51864fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560e5186a3e0 con 0x560e5185e0f0 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e5177c7a0 con 0x560e5185ea10 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x560e5186a560 con 0x560e518675f0 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a639a4740 10 client.0 _ll_register_callbacks cb 0x560e5177eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60116/0 (socket says 172.21.15.7:60116) 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 -- 172.21.15.7:0/4038696623 learned_addr learned my addr 172.21.15.7:0/4038696623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a53e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e51864aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a5a800640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51864fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 -- 172.21.15.7:0/4038696623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 msgr2=0x560e51864fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51864fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 -- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 msgr2=0x560e51864aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e51864aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.513+0000 7f2a59e00640 1 -- 172.21.15.7:0/4038696623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5159d7e0 con 0x560e5185e0f0 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a59e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c005890 tx=0x7f2a4c019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5289a59c4b8b150 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c028020 con 0x560e5185e0f0 2024-04-20T21:40:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f2a4c024620 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a4c0043d0 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a4c033050 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a639a4740 10 client.16500 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.514+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e515fdf70 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c0043d0 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2a4c018070 con 0x560e5185e0f0 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 10 client.16500 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 20 client.16500 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e51877870 0x560e51879c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 4 client.16500 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.515+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e517d0740 con 0x560e51877870 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.516+0000 7f2a53e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e51877870 0x560e51879c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.516+0000 7f2a53e00640 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e51877870 0x560e51879c60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2941014430995bc server_cookie=d84b14a771491d45 in_seq=0 out_seq=0 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.516+0000 7f2a3f400640 10 client.16500 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.517+0000 7f2a3f400640 1 -- 172.21.15.7:0/4038696623 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e517d0740 con 0x560e51877870 2024-04-20T21:40:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.517+0000 7f2a3f400640 10 client.16500 handle_command_reply: tid=0 2024-04-20T21:40:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.517+0000 7f2a639a4740 1 client.16500 shutdown 2024-04-20T21:40:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 msgr2=0x560e51864560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c005890 tx=0x7f2a4c019ba0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 shutdown_connections 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185ea10 0x560e51864aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e51877870 0x560e51879c60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51864fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 --2- 172.21.15.7:0/4038696623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185e0f0 0x560e51864560 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 >> 172.21.15.7:0/4038696623 conn(0x560e51858820 msgr2=0x560e5185a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a3f400640 0 client.16500 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 shutdown_connections 2024-04-20T21:40:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 1 -- 172.21.15.7:0/4038696623 wait complete. 2024-04-20T21:40:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.518+0000 7f2a639a4740 20 client.16500 trim_cache size 0 max 16384 2024-04-20T21:40:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.538+0000 7f2a639a4740 1 Processor -- start 2024-04-20T21:40:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.538+0000 7f2a639a4740 1 -- start start 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.538+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e51866c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.538+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e518732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.538+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a639a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b2590 con 0x560e5185ea10 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a639a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1afe50 con 0x560e5185e0f0 2024-04-20T21:40:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a639a4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1adcc0 con 0x560e518675f0 2024-04-20T21:40:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46726/0 (socket says 172.21.15.7:46726) 2024-04-20T21:40:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a59e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e51866c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 -- 172.21.15.7:0/1997389405 learned_addr learned my addr 172.21.15.7:0/1997389405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a53e00640 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e518732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 -- 172.21.15.7:0/1997389405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 msgr2=0x560e51866c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e51866c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 -- 172.21.15.7:0/1997389405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 msgr2=0x560e518732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e518732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.539+0000 7f2a5a800640 1 -- 172.21.15.7:0/1997389405 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e51495170 con 0x560e518675f0 2024-04-20T21:40:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.540+0000 7f2a5a800640 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a5409cbe0 tx=0x7f2a54010430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efdf2fdf8c06fd4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.540+0000 7f2a4a600640 1 -- 172.21.15.7:0/1997389405 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5409cee0 con 0x560e518675f0 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.540+0000 7f2a4a600640 1 -- 172.21.15.7:0/1997389405 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a540bb070 con 0x560e518675f0 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.540+0000 7f2a4a600640 1 -- 172.21.15.7:0/1997389405 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540ce910 con 0x560e518675f0 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- 172.21.15.7:0/1997389405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 msgr2=0x560e51857880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a5409cbe0 tx=0x7f2a54010430 comp rx=0 tx=0).stop 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- 172.21.15.7:0/1997389405 shutdown_connections 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51857880 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e518732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 --2- 172.21.15.7:0/1997389405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e51866c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- 172.21.15.7:0/1997389405 >> 172.21.15.7:0/1997389405 conn(0x560e51858820 msgr2=0x560e51857170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- 172.21.15.7:0/1997389405 shutdown_connections 2024-04-20T21:40:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- 172.21.15.7:0/1997389405 wait complete. 2024-04-20T21:40:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.541+0000 7f2a639a4740 1 -- start start 2024-04-20T21:40:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 Processor -- start 2024-04-20T21:40:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51861c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1ae310 con 0x560e5185ea10 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1af780 con 0x560e5185e0f0 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1ae9e0 con 0x560e518675f0 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a639a4740 10 client.0 _ll_register_callbacks cb 0x7f2a4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.542+0000 7f2a59e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48066/0 (socket says 172.21.15.7:48066) 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a53e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60130/0 (socket says 172.21.15.7:60130) 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 -- 172.21.15.7:0/2170062393 learned_addr learned my addr 172.21.15.7:0/2170062393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a5a800640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51861c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 -- 172.21.15.7:0/2170062393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 msgr2=0x560e51861c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51861c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 -- 172.21.15.7:0/2170062393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 msgr2=0x560e51861710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 -- 172.21.15.7:0/2170062393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e518e5510 con 0x560e5185e0f0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a53e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a59e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c018500 tx=0x7f2a4c0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79130eec40271a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c015070 con 0x560e5185e0f0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a540bb070 con 0x560e5185e0f0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.543+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f2a4c018970 con 0x560e5185e0f0 2024-04-20T21:40:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a4c028040 con 0x560e5185e0f0 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 10 client.26717 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560e518e6760 con 0x560e5185e0f0 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a540bb070 con 0x560e5185e0f0 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2a4c03f070 con 0x560e5185e0f0 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 10 client.26717 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 10 client.26717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 20 client.26717 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e518e6ea0 0x560e518e92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 4 client.26717 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.544+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e517ca5b0 con 0x560e518e6ea0 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.545+0000 7f2a53e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e518e6ea0 0x560e518e92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.545+0000 7f2a53e00640 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e518e6ea0 0x560e518e92c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8a1b10f5c95c1e server_cookie=64469b231dcd7038 in_seq=0 out_seq=0 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.545+0000 7f2a3f400640 10 client.26717 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.546+0000 7f2a3f400640 1 -- 172.21.15.7:0/2170062393 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e517ca5b0 con 0x560e518e6ea0 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.546+0000 7f2a3f400640 10 client.26717 handle_command_reply: tid=0 2024-04-20T21:40:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.546+0000 7f2a639a4740 1 client.26717 shutdown 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 msgr2=0x560e518611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c018500 tx=0x7f2a4c0096a0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 shutdown_connections 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x560e518e6ea0 0x560e518e92c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560e518675f0 0x560e51861c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560e5185ea10 0x560e51861710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 --2- 172.21.15.7:0/2170062393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e5185e0f0 0x560e518611d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 >> 172.21.15.7:0/2170062393 conn(0x560e51858820 msgr2=0x560e51864d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a3f400640 0 client.26717 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 shutdown_connections 2024-04-20T21:40:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 1 -- 172.21.15.7:0/2170062393 wait complete. 2024-04-20T21:40:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.547+0000 7f2a639a4740 20 client.26717 trim_cache size 0 max 16384 2024-04-20T21:40:59.554 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:59.554 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:40:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "c19f0474-8756-4af1-9cb3-6ede6936fb7e", 2024-04-20T21:40:59.555 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:40:59.555 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 msgr2=0x7f2a400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 0x7f2a400a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x560e518599b0 tx=0x7f2a44005dd0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 msgr2=0x7f2a5c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400efd0 tx=0x7f2a5400c6a0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 shutdown_connections 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2a400a3d20 0x7f2a400a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c109290 0x7f2a5c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a5c19a740 0x7f2a5c19abc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 --2- 172.21.15.7:0/240730080 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2a5c1041c0 0x7f2a5c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 >> 172.21.15.7:0/240730080 conn(0x7f2a5c0ffa00 msgr2=0x7f2a5c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 shutdown_connections 2024-04-20T21:40:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.548+0000 7f2a5b200640 1 -- 172.21.15.7:0/240730080 wait complete. 2024-04-20T21:40:59.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:40:59.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:40:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 -- 172.21.15.7:0/3920409780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c10e3e0 msgr2=0x7fed8c10e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 --2- 172.21.15.7:0/3920409780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c10e3e0 0x7fed8c10e7c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009930 tx=0x7fed7c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:40:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 -- 172.21.15.7:0/3920409780 shutdown_connections 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 --2- 172.21.15.7:0/3920409780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 0x7fed8c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 --2- 172.21.15.7:0/3920409780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c108e40 0x7fed8c115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 --2- 172.21.15.7:0/3920409780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c10e3e0 0x7fed8c10e7c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 -- 172.21.15.7:0/3920409780 >> 172.21.15.7:0/3920409780 conn(0x7fed8c077d70 msgr2=0x7fed8c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 -- 172.21.15.7:0/3920409780 shutdown_connections 2024-04-20T21:40:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.771+0000 7fed8b200640 1 -- 172.21.15.7:0/3920409780 wait complete. 2024-04-20T21:40:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.772+0000 7fed8b200640 1 Processor -- start 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 -- start start 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 0x7fed8c1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c10e3e0 0x7fed8c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c118760 con 0x7fed8c108e40 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1185e0 con 0x7fed8c1084a0 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1188e0 con 0x7fed8c10e3e0 2024-04-20T21:40:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60148/0 (socket says 172.21.15.7:60148) 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed89400640 1 -- 172.21.15.7:0/3097205477 learned_addr learned my addr 172.21.15.7:0/3097205477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed8a800640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c10e3e0 0x7fed8c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.773+0000 7fed89e00640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 0x7fed8c1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 -- 172.21.15.7:0/3097205477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c10e3e0 msgr2=0x7fed8c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c10e3e0 0x7fed8c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 -- 172.21.15.7:0/3097205477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 msgr2=0x7fed8c1a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 0x7fed8c1a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 -- 172.21.15.7:0/3097205477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c19f1e0 con 0x7fed8c108e40 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed89400640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed7400e9f0 tx=0x7fed7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed82a00640 1 -- 172.21.15.7:0/3097205477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7400cd80 con 0x7fed8c108e40 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c19f4d0 con 0x7fed8c108e40 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.774+0000 7fed82a00640 1 -- 172.21.15.7:0/3097205477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed74010910 con 0x7fed8c108e40 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.775+0000 7fed82a00640 1 -- 172.21.15.7:0/3097205477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740188f0 con 0x7fed8c108e40 2024-04-20T21:40:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.775+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1ab770 con 0x7fed8c108e40 2024-04-20T21:40:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.779+0000 7fed93d6d740 1 Processor -- start 2024-04-20T21:40:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.779+0000 7fed93d6d740 1 -- start start 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.779+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 0x55d348c20340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed93d6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d348b7d150 con 0x55d348c17380 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed93d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d348b23370 con 0x55d348c20880 2024-04-20T21:40:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed93d6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d348b79110 con 0x55d348c17ca0 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed8a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed8a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60150/0 (socket says 172.21.15.7:60150) 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed8a800640 1 -- 172.21.15.7:0/4202637952 learned_addr learned my addr 172.21.15.7:0/4202637952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 0x55d348c20340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed82a00640 1 -- 172.21.15.7:0/3097205477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fed74010ac0 con 0x7fed8c108e40 2024-04-20T21:40:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 -- 172.21.15.7:0/4202637952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 msgr2=0x55d348c22c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:40:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c22c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 -- 172.21.15.7:0/4202637952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 msgr2=0x55d348c17760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed89e00640 1 -- 172.21.15.7:0/4202637952 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3489597d0 con 0x55d348c17ca0 2024-04-20T21:40:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.780+0000 7fed8a800640 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.781+0000 7fed89e00640 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 0x55d348c20340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed7c036040 tx=0x7fed7c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d995f9a0a6b9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.781+0000 7fed7b400640 1 -- 172.21.15.7:0/4202637952 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c045070 con 0x55d348c17ca0 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.781+0000 7fed7b400640 1 -- 172.21.15.7:0/4202637952 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed7c02c650 con 0x55d348c17ca0 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.781+0000 7fed7b400640 1 -- 172.21.15.7:0/4202637952 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c004b60 con 0x55d348c17ca0 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed82a00640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed82a00640 1 -- 172.21.15.7:0/3097205477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fed740d0ba0 con 0x7fed8c108e40 2024-04-20T21:40:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed8a800640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 -- 172.21.15.7:0/4202637952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 msgr2=0x55d348c20340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 0x55d348c20340 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed7c036040 tx=0x7fed7c009560 comp rx=0 tx=0).stop 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 -- 172.21.15.7:0/4202637952 shutdown_connections 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c22c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17ca0 0x55d348c20340 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4202637952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 -- 172.21.15.7:0/4202637952 >> 172.21.15.7:0/4202637952 conn(0x55d348c11ab0 msgr2=0x55d348c11e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.782+0000 7fed93d6d740 1 -- 172.21.15.7:0/4202637952 shutdown_connections 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.783+0000 7fed93d6d740 1 -- 172.21.15.7:0/4202637952 wait complete. 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.783+0000 7fed8a800640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed84009fd0 tx=0x7fed84009500 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.783+0000 7fed93d6d740 1 -- start start 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.783+0000 7fed93d6d740 1 Processor -- start 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17380 0x55d348c1d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d348c23610 con 0x55d348c17ca0 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d348b36740 con 0x55d348c17380 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55d348c23790 con 0x55d348c20880 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed93d6d740 10 client.0 _ll_register_callbacks cb 0x55d348b31b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed8a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60160/0 (socket says 172.21.15.7:60160) 2024-04-20T21:40:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed89400640 1 -- 172.21.15.7:0/1075454081 learned_addr learned my addr 172.21.15.7:0/1075454081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.784+0000 7fed89e00640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17380 0x55d348c1d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 -- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17380 msgr2=0x55d348c1d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17380 0x55d348c1d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 -- 172.21.15.7:0/1075454081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 msgr2=0x55d348c1de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 -- 172.21.15.7:0/1075454081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34888f3f0 con 0x55d348c20880 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed89400640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed8a800640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c1e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed8400fb50 tx=0x7fed84004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e056aad8b27de455 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8400fc40 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed7c02c650 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.785+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fed84017030 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed8400fde0 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 10 client.26683 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d34884f0f0 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed7c02c650 con 0x55d348c20880 2024-04-20T21:40:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fed8402f030 con 0x55d348c20880 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 10 client.26683 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 20 client.26683 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c30c10 0x55d348c33030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 4 client.26683 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.786+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d348b8ab70 con 0x55d348c30c10 2024-04-20T21:40:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.787+0000 7fed89e00640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c30c10 0x55d348c33030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.787+0000 7fed89e00640 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c30c10 0x55d348c33030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecc07ad0eebd826 server_cookie=825c8cbd0ac7b6e5 in_seq=0 out_seq=0 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.787+0000 7fed6fe00640 10 client.26683 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed6fe00640 1 -- 172.21.15.7:0/1075454081 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d348b8ab70 con 0x55d348c30c10 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed6fe00640 10 client.26683 handle_command_reply: tid=0 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 client.26683 shutdown 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 msgr2=0x55d348c1e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c1e340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fed8400fb50 tx=0x7fed84004ab0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 shutdown_connections 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c30c10 0x55d348c33030 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c1e340 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 --2- 172.21.15.7:0/1075454081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17380 0x55d348c1d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed6fe00640 0 client.26683 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.788+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 >> 172.21.15.7:0/1075454081 conn(0x55d348c11ab0 msgr2=0x55d348c13800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.789+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 shutdown_connections 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.789+0000 7fed93d6d740 1 -- 172.21.15.7:0/1075454081 wait complete. 2024-04-20T21:40:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.789+0000 7fed93d6d740 20 client.26683 trim_cache size 0 max 16384 2024-04-20T21:40:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.808+0000 7fed93d6d740 1 Processor -- start 2024-04-20T21:40:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.808+0000 7fed93d6d740 1 -- start start 2024-04-20T21:40:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17380 0x55d348c1fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b6970 con 0x55d348c17ca0 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b4230 con 0x55d348c20880 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed93d6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b20a0 con 0x55d348c17380 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:48102/0 (socket says 172.21.15.7:48102) 2024-04-20T21:40:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed8a800640 1 -- 172.21.15.7:0/4013126654 learned_addr learned my addr 172.21.15.7:0/4013126654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed89e00640 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17380 0x55d348c1fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed8a800640 1 -- 172.21.15.7:0/4013126654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17380 msgr2=0x55d348c1fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.809+0000 7fed8a800640 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17380 0x55d348c1fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed8a800640 1 -- 172.21.15.7:0/4013126654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 msgr2=0x55d348c1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed8a800640 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed8a800640 1 -- 172.21.15.7:0/4013126654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d348c2f810 con 0x55d348c20880 2024-04-20T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed8a800640 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fed84004a80 tx=0x7fed8400fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b898252b37030d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed7aa00640 1 -- 172.21.15.7:0/4013126654 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8403e070 con 0x55d348c20880 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed7aa00640 1 -- 172.21.15.7:0/4013126654 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed8401ee50 con 0x55d348c20880 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.810+0000 7fed7aa00640 1 -- 172.21.15.7:0/4013126654 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84003e10 con 0x55d348c20880 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 -- 172.21.15.7:0/4013126654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 msgr2=0x55d348c2c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fed84004a80 tx=0x7fed8400fc40 comp rx=0 tx=0).stop 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 -- 172.21.15.7:0/4013126654 shutdown_connections 2024-04-20T21:40:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c20880 0x55d348c2c4d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17ca0 0x55d348c1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 --2- 172.21.15.7:0/4013126654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c17380 0x55d348c1fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 -- 172.21.15.7:0/4013126654 >> 172.21.15.7:0/4013126654 conn(0x55d348c11ab0 msgr2=0x55d348c13790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 -- 172.21.15.7:0/4013126654 shutdown_connections 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.811+0000 7fed93d6d740 1 -- 172.21.15.7:0/4013126654 wait complete. 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 -- start start 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 Processor -- start 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 0x55d348c28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b26f0 con 0x55d348c17380 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b3b60 con 0x55d348c17ca0 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c1b2dc0 con 0x55d348c20880 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.812+0000 7fed93d6d740 10 client.0 _ll_register_callbacks cb 0x7fed84009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 0x55d348c28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60178/0 (socket says 172.21.15.7:60178) 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89e00640 1 -- 172.21.15.7:0/2300096296 learned_addr learned my addr 172.21.15.7:0/2300096296 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed8a800640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 -- 172.21.15.7:0/2300096296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 msgr2=0x55d348c23e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c23e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 -- 172.21.15.7:0/2300096296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 msgr2=0x55d348c28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 -- 172.21.15.7:0/2300096296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d348c246b0 con 0x55d348c17ca0 2024-04-20T21:40:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89e00640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.813+0000 7fed89400640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 0x55d348c28ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fed740ce3c0 tx=0x7fed74098790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69bb3fcde1c8903f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed740b6060 con 0x55d348c17ca0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fed7409c260 con 0x55d348c17ca0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed8401ee50 con 0x55d348c17ca0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed740a57d0 con 0x55d348c17ca0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed93d6d740 10 client.26732 fetch_fsmap learned FSMap version 27 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d348c96de0 con 0x55d348c17ca0 2024-04-20T21:40:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed8401ee50 con 0x55d348c17ca0 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.814+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fed7409bc40 con 0x55d348c17ca0 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 10 client.26732 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 10 client.26732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 20 client.26732 populate_metadata read hostname 'smithi007' 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c97560 0x55d348c99980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 4 client.26732 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d348b849e0 con 0x55d348c97560 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed89e00640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c97560 0x55d348c99980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:40:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed89e00640 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c97560 0x55d348c99980 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0b898fa0bf106 server_cookie=df209f1130d764ec in_seq=0 out_seq=0 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.815+0000 7fed6fe00640 10 client.26732 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed6fe00640 1 -- 172.21.15.7:0/2300096296 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d348b849e0 con 0x55d348c97560 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed6fe00640 10 client.26732 handle_command_reply: tid=0 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 client.26732 shutdown 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 msgr2=0x55d348c28ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 0x55d348c28ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fed740ce3c0 tx=0x7fed74098790 comp rx=0 tx=0).stop 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 shutdown_connections 2024-04-20T21:40:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d348c17ca0 0x55d348c28ec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55d348c97560 0x55d348c99980 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d348c20880 0x55d348c23e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 --2- 172.21.15.7:0/2300096296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d348c17380 0x55d348c28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 >> 172.21.15.7:0/2300096296 conn(0x55d348c11ab0 msgr2=0x55d348c1d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed6fe00640 0 client.26732 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 shutdown_connections 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 1 -- 172.21.15.7:0/2300096296 wait complete. 2024-04-20T21:40:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.816+0000 7fed93d6d740 20 client.26732 trim_cache size 0 max 16384 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "c19f0474-8756-4af1-9cb3-6ede6936fb7e": { 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "c19f0474-8756-4af1-9cb3-6ede6936fb7e", 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:40:59.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:40:59.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:40:59.825 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:40:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 msgr2=0x7fed700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 0x7fed700a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fed84009fd0 tx=0x7fed84009500 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 msgr2=0x7fed8c1a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fed7400e9f0 tx=0x7fed7400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 shutdown_connections 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed8c10e3e0 0x7fed8c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c108e40 0x7fed8c1a4df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fed700a3d20 0x7fed700a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 --2- 172.21.15.7:0/3097205477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed8c1084a0 0x7fed8c1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:40:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 >> 172.21.15.7:0/3097205477 conn(0x7fed8c077d70 msgr2=0x7fed8c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:40:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 shutdown_connections 2024-04-20T21:40:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:40:59.817+0000 7fed8b200640 1 -- 172.21.15.7:0/3097205477 wait complete. 2024-04-20T21:40:59.835 INFO:tasks.cephfs.filesystem:scrub status for tag:c19f0474-8756-4af1-9cb3-6ede6936fb7e - {'path': '/', 'tag': 'c19f0474-8756-4af1-9cb3-6ede6936fb7e', 'options': 'recursive,force'} 2024-04-20T21:41:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: pgmap v235: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:00.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:00 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: pgmap v235: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:00.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:00.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:00 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: pgmap v235: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:00.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:00 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:01 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:41:01.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:01 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:01 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:01.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:01 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:01 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:41:01.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:01 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:01 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:01.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:01 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:01 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:41:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:01 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:01 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:01.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:01 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:02.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:02 smithi007 ceph-mon[16867]: pgmap v236: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:02.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:02 smithi016 ceph-mon[21409]: pgmap v236: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:02.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:02 smithi186 ceph-mon[22157]: pgmap v236: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:04 smithi007 ceph-mon[16867]: pgmap v237: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:04 smithi016 ceph-mon[21409]: pgmap v237: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:04.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:04 smithi186 ceph-mon[22157]: pgmap v237: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:06.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:06 smithi186 ceph-mon[22157]: pgmap v238: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-04-20T21:41:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:06 smithi007 ceph-mon[16867]: pgmap v238: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-04-20T21:41:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:06 smithi016 ceph-mon[21409]: pgmap v238: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 23 op/s 2024-04-20T21:41:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:07 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:07 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:07 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:07 smithi007 ceph-mon[16867]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:41:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:07 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:07 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:07 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:07 smithi016 ceph-mon[21409]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:41:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:07 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:07 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:07 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:07 smithi186 ceph-mon[22157]: osdmap e92: 12 total, 12 up, 12 in 2024-04-20T21:41:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:08 smithi007 ceph-mon[16867]: pgmap v240: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:08 smithi016 ceph-mon[21409]: pgmap v240: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:08.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:08 smithi186 ceph-mon[22157]: pgmap v240: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:10 smithi007 ceph-mon[16867]: pgmap v241: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:10.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:10 smithi016 ceph-mon[21409]: pgmap v241: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:10.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:10 smithi186 ceph-mon[22157]: pgmap v241: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:12 smithi007 ceph-mon[16867]: pgmap v242: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:12.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:12 smithi016 ceph-mon[21409]: pgmap v242: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:12 smithi186 ceph-mon[22157]: pgmap v242: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:14 smithi007 ceph-mon[16867]: pgmap v243: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:14 smithi007 ceph-mon[16867]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:41:14.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:14 smithi016 ceph-mon[21409]: pgmap v243: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:14.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:14 smithi016 ceph-mon[21409]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:41:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:14 smithi186 ceph-mon[22157]: pgmap v243: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 27 op/s 2024-04-20T21:41:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:14 smithi186 ceph-mon[22157]: osdmap e93: 12 total, 12 up, 12 in 2024-04-20T21:41:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:16 smithi007 ceph-mon[16867]: pgmap v245: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 465 B/s wr, 7 op/s 2024-04-20T21:41:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:16 smithi016 ceph-mon[21409]: pgmap v245: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 465 B/s wr, 7 op/s 2024-04-20T21:41:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:16 smithi186 ceph-mon[22157]: pgmap v245: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 465 B/s wr, 7 op/s 2024-04-20T21:41:18.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:18 smithi007 ceph-mon[16867]: pgmap v246: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-20T21:41:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:18 smithi016 ceph-mon[21409]: pgmap v246: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-20T21:41:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:18 smithi186 ceph-mon[22157]: pgmap v246: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-20T21:41:20.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:20 smithi016 ceph-mon[21409]: pgmap v247: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:20.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:20 smithi186 ceph-mon[22157]: pgmap v247: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:21.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:20 smithi007 ceph-mon[16867]: pgmap v247: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:22.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:22 smithi016 ceph-mon[21409]: pgmap v248: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:22.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:22 smithi186 ceph-mon[22157]: pgmap v248: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:23.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:22 smithi007 ceph-mon[16867]: pgmap v248: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:24.835 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:24 smithi007 ceph-mon[16867]: pgmap v249: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:24.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:24 smithi016 ceph-mon[21409]: pgmap v249: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:24 smithi186 ceph-mon[22157]: pgmap v249: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:41:26.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:26 smithi016 ceph-mon[21409]: pgmap v250: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:41:26.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:26 smithi186 ceph-mon[22157]: pgmap v250: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:41:27.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:26 smithi007 ceph-mon[16867]: pgmap v250: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:41:28.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:28 smithi016 ceph-mon[21409]: pgmap v251: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:28.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:28 smithi186 ceph-mon[22157]: pgmap v251: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:29.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:28 smithi007 ceph-mon[16867]: pgmap v251: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:29.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:29.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:29.916 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:29 smithi007 ceph-mon[16867]: pgmap v252: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:29.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:29 smithi016 ceph-mon[21409]: pgmap v252: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:29.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:29 smithi186 ceph-mon[22157]: pgmap v252: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:41:30.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 -- 172.21.15.7:0/246474983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 msgr2=0x7f8d94115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 --2- 172.21.15.7:0/246474983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d94115210 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009cd0 tx=0x7f8d8c02c0f0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 -- 172.21.15.7:0/246474983 shutdown_connections 2024-04-20T21:41:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 --2- 172.21.15.7:0/246474983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d94115210 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 --2- 172.21.15.7:0/246474983 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d941084a0 0x7f8d94108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 --2- 172.21.15.7:0/246474983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d9410e3e0 0x7f8d9410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 -- 172.21.15.7:0/246474983 >> 172.21.15.7:0/246474983 conn(0x7f8d94077d70 msgr2=0x7f8d94106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 -- 172.21.15.7:0/246474983 shutdown_connections 2024-04-20T21:41:30.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.058+0000 7f8d93200640 1 -- 172.21.15.7:0/246474983 wait complete. 2024-04-20T21:41:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.059+0000 7f8d93200640 1 Processor -- start 2024-04-20T21:41:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 -- start start 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 0x7f8d941a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d941a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94118710 con 0x7f8d94108e40 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94118590 con 0x7f8d941084a0 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d93200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94118890 con 0x7f8d9410e3e0 2024-04-20T21:41:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d92800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 0x7f8d941a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d92800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40184/0 (socket says 172.21.15.7:40184) 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d92800640 1 -- 172.21.15.7:0/878532135 learned_addr learned my addr 172.21.15.7:0/878532135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.060+0000 7f8d91400640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d941a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 -- 172.21.15.7:0/878532135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 msgr2=0x7f8d941a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 0x7f8d941a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 -- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 msgr2=0x7f8d941a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d941a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 -- 172.21.15.7:0/878532135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9419f230 con 0x7f8d9410e3e0 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d91400640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d941a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d91e00640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 0x7f8d941a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d92800640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005e30 tx=0x7f8d8c032ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.061+0000 7f8d8aa00640 1 -- 172.21.15.7:0/878532135 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c041020 con 0x7f8d9410e3e0 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.062+0000 7f8d8aa00640 1 -- 172.21.15.7:0/878532135 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8c034cd0 con 0x7f8d9410e3e0 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.062+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9419f4c0 con 0x7f8d9410e3e0 2024-04-20T21:41:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.062+0000 7f8d8aa00640 1 -- 172.21.15.7:0/878532135 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c039040 con 0x7f8d9410e3e0 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.062+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d941ab7c0 con 0x7f8d9410e3e0 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.065+0000 7f8d9b7e0740 1 Processor -- start 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.065+0000 7f8d8aa00640 1 -- 172.21.15.7:0/878532135 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8d8c044020 con 0x7f8d9410e3e0 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.065+0000 7f8d9b7e0740 1 -- start start 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f404163c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f4040be150 con 0x55f404158380 2024-04-20T21:41:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f404064370 con 0x55f404158ca0 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f4040ba110 con 0x55f404161880 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57688/0 (socket says 172.21.15.7:57688) 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54514/0 (socket says 172.21.15.7:54514) 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 -- 172.21.15.7:0/2994633023 learned_addr learned my addr 172.21.15.7:0/2994633023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91400640 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f404163c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 -- 172.21.15.7:0/2994633023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 msgr2=0x55f404163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f404163c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 -- 172.21.15.7:0/2994633023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 msgr2=0x55f404158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.066+0000 7f8d91e00640 1 -- 172.21.15.7:0/2994633023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f403e9a7d0 con 0x55f404158ca0 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d8aa00640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 0x7f8d780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d91e00640 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8d84011470 tx=0x7f8d84011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64c1c7f6498e7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d8aa00640 1 -- 172.21.15.7:0/878532135 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f8d8c0f23c0 con 0x7f8d9410e3e0 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d7f000640 1 -- 172.21.15.7:0/2994633023 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84005020 con 0x55f404158ca0 2024-04-20T21:41:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d91400640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 0x7f8d780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d7f000640 1 -- 172.21.15.7:0/2994633023 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d840047a0 con 0x55f404158ca0 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.067+0000 7f8d7f000640 1 -- 172.21.15.7:0/2994633023 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400f880 con 0x55f404158ca0 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d91400640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 0x7f8d780a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55f404153c40 tx=0x7f8d80015040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/2994633023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 msgr2=0x55f404161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8d84011470 tx=0x7f8d84011930 comp rx=0 tx=0).stop 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/2994633023 shutdown_connections 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404161340 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f404163c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/2994633023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/2994633023 >> 172.21.15.7:0/2994633023 conn(0x55f404152ab0 msgr2=0x55f404152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/2994633023 shutdown_connections 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.068+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/2994633023 wait complete. 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 -- start start 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 Processor -- start 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f40415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f40415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f404164670 con 0x55f404161880 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f404077740 con 0x55f404158ca0 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55f4041647f0 con 0x55f404158380 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.069+0000 7f8d9b7e0740 10 client.0 _ll_register_callbacks cb 0x55f404072b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57696/0 (socket says 172.21.15.7:57696) 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f40415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 -- 172.21.15.7:0/1955421507 learned_addr learned my addr 172.21.15.7:0/1955421507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f40415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 -- 172.21.15.7:0/1955421507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 msgr2=0x55f40415e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f40415e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 -- 172.21.15.7:0/1955421507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 msgr2=0x55f40415f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f40415f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 -- 172.21.15.7:0/1955421507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f403dd03f0 con 0x55f404158ca0 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d92800640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f40415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d91400640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011bd0 tx=0x7f8d80005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f81d0c8e9929bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.070+0000 7f8d73400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80022dd0 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f8d80023420 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d840047a0 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d80023620 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d9b7e0740 10 client.26744 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f403d900f0 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.071+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d840047a0 con 0x55f404158ca0 2024-04-20T21:41:30.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8d80032070 con 0x55f404158ca0 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 10 client.26744 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 10 client.26744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 20 client.26744 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f404171cd0 0x55f404174150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 4 client.26744 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4040cbb70 con 0x55f404171cd0 2024-04-20T21:41:30.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.072+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f404171cd0 0x55f404174150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.073+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f404171cd0 0x55f404174150 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6671c36e53e5791f server_cookie=84785dcde2cd9953 in_seq=0 out_seq=0 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.073+0000 7f8d73400640 10 client.26744 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d73400640 1 -- 172.21.15.7:0/1955421507 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4040cbb70 con 0x55f404171cd0 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d73400640 10 client.26744 handle_command_reply: tid=0 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 client.26744 shutdown 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 msgr2=0x55f40415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8d80011bd0 tx=0x7f8d80005f90 comp rx=0 tx=0).stop 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 shutdown_connections 2024-04-20T21:41:30.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40415ed90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f404171cd0 0x55f404174150 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f40415f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1955421507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f40415e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d73400640 0 client.26744 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 >> 172.21.15.7:0/1955421507 conn(0x55f404152ab0 msgr2=0x55f404154800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 shutdown_connections 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1955421507 wait complete. 2024-04-20T21:41:30.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.074+0000 7f8d9b7e0740 20 client.26744 trim_cache size 0 max 16384 2024-04-20T21:41:30.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 Processor -- start 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 -- start start 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404160f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f40415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b6890 con 0x55f404158380 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b4150 con 0x55f404158ca0 2024-04-20T21:41:30.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b1fc0 con 0x55f404161880 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d91e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.094+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57706/0 (socket says 172.21.15.7:57706) 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 -- 172.21.15.7:0/1148034390 learned_addr learned my addr 172.21.15.7:0/1148034390 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d92800640 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f40415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 -- 172.21.15.7:0/1148034390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 msgr2=0x55f40415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f40415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 -- 172.21.15.7:0/1148034390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 msgr2=0x55f404160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 -- 172.21.15.7:0/1148034390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4041528e0 con 0x55f404158ca0 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d91400640 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8d8001b040 tx=0x7f8d80009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01a629ae81958a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.095+0000 7f8d7e600640 1 -- 172.21.15.7:0/1148034390 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80038030 con 0x55f404158ca0 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d7e600640 1 -- 172.21.15.7:0/1148034390 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80023420 con 0x55f404158ca0 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d7e600640 1 -- 172.21.15.7:0/1148034390 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800237c0 con 0x55f404158ca0 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1148034390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 msgr2=0x55f40416d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8d8001b040 tx=0x7f8d80009690 comp rx=0 tx=0).stop 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1148034390 shutdown_connections 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f40416d560 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404161880 0x55f40415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1148034390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404158380 0x55f404160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1148034390 >> 172.21.15.7:0/1148034390 conn(0x55f404152ab0 msgr2=0x55f40415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1148034390 shutdown_connections 2024-04-20T21:41:30.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.096+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1148034390 wait complete. 2024-04-20T21:41:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 -- start start 2024-04-20T21:41:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 Processor -- start 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f404169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f404165a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b2610 con 0x55f404161880 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b3a80 con 0x55f404158ca0 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941b2ce0 con 0x55f404158380 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.097+0000 7f8d9b7e0740 10 client.0 _ll_register_callbacks cb 0x7f8d80009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f404165a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57708/0 (socket says 172.21.15.7:57708) 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 -- 172.21.15.7:0/1220745505 learned_addr learned my addr 172.21.15.7:0/1220745505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f404169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 -- 172.21.15.7:0/1220745505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 msgr2=0x55f404169fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f404169fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 -- 172.21.15.7:0/1220745505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 msgr2=0x55f404165a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f404165a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 -- 172.21.15.7:0/1220745505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f404170820 con 0x55f404158ca0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d91400640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d80020b40 tx=0x7f8d80009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210ebee219f17889 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.098+0000 7f8d73400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8004a060 con 0x55f404158ca0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f8d80020620 con 0x55f404158ca0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80023420 con 0x55f404158ca0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8d80038030 con 0x55f404158ca0 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d9b7e0740 10 client.26750 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4041d8270 con 0x55f404158ca0 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.099+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80023420 con 0x55f404158ca0 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8d80032050 con 0x55f404158ca0 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 10 client.26750 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 10 client.26750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 20 client.26750 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f4041d8930 0x55f4041dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 4 client.26750 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4040c59e0 con 0x55f4041d8930 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f4041d8930 0x55f4041dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d91e00640 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f4041d8930 0x55f4041dad50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7085f0dd10fab44 server_cookie=3d8f4f7d211af539 in_seq=0 out_seq=0 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.100+0000 7f8d73400640 10 client.26750 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d73400640 1 -- 172.21.15.7:0/1220745505 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4040c59e0 con 0x55f4041d8930 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d73400640 10 client.26750 handle_command_reply: tid=0 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 client.26750 shutdown 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 msgr2=0x55f404165330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d80020b40 tx=0x7f8d80009420 comp rx=0 tx=0).stop 2024-04-20T21:41:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 shutdown_connections 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f404158ca0 0x55f404165330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55f4041d8930 0x55f4041dad50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f404161880 0x55f404165a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 --2- 172.21.15.7:0/1220745505 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f404158380 0x55f404169fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d73400640 0 client.26750 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 >> 172.21.15.7:0/1220745505 conn(0x55f404152ab0 msgr2=0x55f404151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 shutdown_connections 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 1 -- 172.21.15.7:0/1220745505 wait complete. 2024-04-20T21:41:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.101+0000 7f8d9b7e0740 20 client.26750 trim_cache size 0 max 16384 2024-04-20T21:41:30.109 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:41:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:41:30.109 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:41:30.109 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:41:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 msgr2=0x7f8d780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 0x7f8d780a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55f404153c40 tx=0x7f8d80015040 comp rx=0 tx=0).stop 2024-04-20T21:41:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 msgr2=0x7f8d9419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005e30 tx=0x7f8d8c032ea0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 shutdown_connections 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d9410e3e0 0x7f8d9419eab0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d94108e40 0x7f8d941a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d780a3da0 0x7f8d780a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 --2- 172.21.15.7:0/878532135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d941084a0 0x7f8d941a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 >> 172.21.15.7:0/878532135 conn(0x7f8d94077d70 msgr2=0x7f8d94078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 shutdown_connections 2024-04-20T21:41:30.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.102+0000 7f8d93200640 1 -- 172.21.15.7:0/878532135 wait complete. 2024-04-20T21:41:30.119 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:41:30.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:41:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.325+0000 7f557fe00640 1 -- 172.21.15.7:0/1631836585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 msgr2=0x7f55801128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.325+0000 7f557fe00640 1 --2- 172.21.15.7:0/1631836585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f55801128f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f556802bcc0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 -- 172.21.15.7:0/1631836585 shutdown_connections 2024-04-20T21:41:30.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 --2- 172.21.15.7:0/1631836585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f55801128f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 --2- 172.21.15.7:0/1631836585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580112e30 0x7f5580115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 --2- 172.21.15.7:0/1631836585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580074300 0x7f55800746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 -- 172.21.15.7:0/1631836585 >> 172.21.15.7:0/1631836585 conn(0x7f55800fd950 msgr2=0x7f55800ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 -- 172.21.15.7:0/1631836585 shutdown_connections 2024-04-20T21:41:30.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.326+0000 7f557fe00640 1 -- 172.21.15.7:0/1631836585 wait complete. 2024-04-20T21:41:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 Processor -- start 2024-04-20T21:41:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 -- start start 2024-04-20T21:41:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580074300 0x7f558019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580112e30 0x7f55801a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5580118860 con 0x7f5580112e30 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55801186e0 con 0x7f5580074c20 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.327+0000 7f557fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f55801189e0 con 0x7f5580074300 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580112e30 0x7f55801a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57736/0 (socket says 172.21.15.7:57736) 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 -- 172.21.15.7:0/3950977214 learned_addr learned my addr 172.21.15.7:0/3950977214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557ea00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580074300 0x7f558019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 -- 172.21.15.7:0/3950977214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580074300 msgr2=0x7f558019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580074300 0x7f558019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 -- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580112e30 msgr2=0x7f55801a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580112e30 0x7f55801a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.328+0000 7f557e000640 1 -- 172.21.15.7:0/3950977214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55801a3d70 con 0x7f5580074c20 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f557e000640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5568000c00 tx=0x7f55680043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568042020 con 0x7f5580074c20 2024-04-20T21:41:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568005000 con 0x7f5580074c20 2024-04-20T21:41:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55801a4000 con 0x7f5580074c20 2024-04-20T21:41:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568003b20 con 0x7f5580074c20 2024-04-20T21:41:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.329+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55801ab8f0 con 0x7f5580074c20 2024-04-20T21:41:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.330+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558004fd60 con 0x7f5580074c20 2024-04-20T21:41:30.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.332+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5568045020 con 0x7f5580074c20 2024-04-20T21:41:30.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.334+0000 7f5577400640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 0x7f55640a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.334+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f55680f27c0 con 0x7f5580074c20 2024-04-20T21:41:30.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.334+0000 7f557ea00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 0x7f55640a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.335+0000 7f557ea00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 0x7f55640a61d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f557000bbe0 tx=0x7f557000b390 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.336+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f556804c020 con 0x7f5580074c20 2024-04-20T21:41:30.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.585+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55801052c0 con 0x7f5580074c20 2024-04-20T21:41:30.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.586+0000 7f5577400640 1 -- 172.21.15.7:0/3950977214 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f55680bef30 con 0x7f5580074c20 2024-04-20T21:41:30.585 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:41:30.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:41:30.585 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 msgr2=0x7f55640a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 0x7f55640a61d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f557000bbe0 tx=0x7f557000b390 comp rx=0 tx=0).stop 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 msgr2=0x7f558019f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5568000c00 tx=0x7f55680043c0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 shutdown_connections 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f55640a3d10 0x7f55640a61d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5580074c20 0x7f558019f2c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580112e30 0x7f55801a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 --2- 172.21.15.7:0/3950977214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5580074300 0x7f558019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 >> 172.21.15.7:0/3950977214 conn(0x7f55800fd950 msgr2=0x7f55800fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 shutdown_connections 2024-04-20T21:41:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.590+0000 7f557fe00640 1 -- 172.21.15.7:0/3950977214 wait complete. 2024-04-20T21:41:30.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 -- 172.21.15.7:0/917276330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a8104ba0 msgr2=0x7f67a8108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 --2- 172.21.15.7:0/917276330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a8104ba0 0x7f67a8108be0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6790009900 tx=0x7f679002bc40 comp rx=0 tx=0).stop 2024-04-20T21:41:30.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 -- 172.21.15.7:0/917276330 shutdown_connections 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 --2- 172.21.15.7:0/917276330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a8104ba0 0x7f67a8108be0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 --2- 172.21.15.7:0/917276330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 0x7f67a8110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 --2- 172.21.15.7:0/917276330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 -- 172.21.15.7:0/917276330 >> 172.21.15.7:0/917276330 conn(0x7f67a80ffa50 msgr2=0x7f67a8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.829+0000 7f67ade00640 1 -- 172.21.15.7:0/917276330 shutdown_connections 2024-04-20T21:41:30.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.830+0000 7f67ade00640 1 -- 172.21.15.7:0/917276330 wait complete. 2024-04-20T21:41:30.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 Processor -- start 2024-04-20T21:41:30.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 -- start start 2024-04-20T21:41:30.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 0x7f67a81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a819a700 0x7f67a819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8114380 con 0x7f67a81041f0 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8114200 con 0x7f67a819a700 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67ade00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8114500 con 0x7f67a81092c0 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54594/0 (socket says 172.21.15.7:54594) 2024-04-20T21:41:30.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.831+0000 7f67aca00640 1 -- 172.21.15.7:0/3465688782 learned_addr learned my addr 172.21.15.7:0/3465688782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f679fe00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 0x7f67a81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67ad400640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a819a700 0x7f67a819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 -- 172.21.15.7:0/3465688782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 msgr2=0x7f67a81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 0x7f67a81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 -- 172.21.15.7:0/3465688782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a819a700 msgr2=0x7f67a819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a819a700 0x7f67a819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 -- 172.21.15.7:0/3465688782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a819b300 con 0x7f67a81041f0 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.832+0000 7f67aca00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f679800d950 tx=0x7f679800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.833+0000 7f67a6000640 1 -- 172.21.15.7:0/3465688782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798017070 con 0x7f67a81041f0 2024-04-20T21:41:30.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.833+0000 7f67a6000640 1 -- 172.21.15.7:0/3465688782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798009c30 con 0x7f67a81041f0 2024-04-20T21:41:30.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.833+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a81a73e0 con 0x7f67a81041f0 2024-04-20T21:41:30.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.833+0000 7f67a6000640 1 -- 172.21.15.7:0/3465688782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679800f6b0 con 0x7f67a81041f0 2024-04-20T21:41:30.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.833+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81a7790 con 0x7f67a81041f0 2024-04-20T21:41:30.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.836+0000 7f67b202e740 1 Processor -- start 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.836+0000 7f67b202e740 1 -- start start 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67a6000640 1 -- 172.21.15.7:0/3465688782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f67980109d0 con 0x7f67a81041f0 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0170 0x56357e9d0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56357e935050 con 0x56357e9d0a90 2024-04-20T21:41:30.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56357e8db490 con 0x56357e9d9670 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67b202e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56357e931010 con 0x56357e9d0170 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f679fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f67ad400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0170 0x56357e9d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f679fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57768/0 (socket says 172.21.15.7:57768) 2024-04-20T21:41:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.837+0000 7f679fe00640 1 -- 172.21.15.7:0/4177446781 learned_addr learned my addr 172.21.15.7:0/4177446781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 -- 172.21.15.7:0/4177446781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0170 msgr2=0x56357e9d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0170 0x56357e9d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 -- 172.21.15.7:0/4177446781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 msgr2=0x56357e9d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9d9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 -- 172.21.15.7:0/4177446781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357e8e3070 con 0x56357e9d9670 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679fe00640 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f679002efd0 tx=0x7f6790000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb789e28c9d034e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679dc00640 1 -- 172.21.15.7:0/4177446781 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790042070 con 0x56357e9d9670 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679dc00640 1 -- 172.21.15.7:0/4177446781 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6790004240 con 0x56357e9d9670 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.838+0000 7f679dc00640 1 -- 172.21.15.7:0/4177446781 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790004f40 con 0x56357e9d9670 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67a6000640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 0x7f678c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67a6000640 1 -- 172.21.15.7:0/3465688782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f67980d03c0 con 0x7f67a81041f0 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67ad400640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 0x7f678c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 -- 172.21.15.7:0/4177446781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 msgr2=0x56357e9dba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f679002efd0 tx=0x7f6790000bc0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 -- 172.21.15.7:0/4177446781 shutdown_connections 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9dba60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 --2- 172.21.15.7:0/4177446781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0170 0x56357e9d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 -- 172.21.15.7:0/4177446781 >> 172.21.15.7:0/4177446781 conn(0x56357e9ca8a0 msgr2=0x56357e9cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.839+0000 7f67b202e740 1 -- 172.21.15.7:0/4177446781 shutdown_connections 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.840+0000 7f67b202e740 1 -- 172.21.15.7:0/4177446781 wait complete. 2024-04-20T21:41:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.840+0000 7f67ad400640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 0x7f678c0a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0009600 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.840+0000 7f67b202e740 1 -- start start 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.840+0000 7f67b202e740 1 Processor -- start 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 0x56357e9d6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 0x56357e9d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56357e9dc400 con 0x56357e9d0170 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56357e8ee860 con 0x56357e9d9670 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56357e9dc580 con 0x56357e9d0a90 2024-04-20T21:41:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67b202e740 10 client.0 _ll_register_callbacks cb 0x56357e8bcfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 0x56357e9d6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57782/0 (socket says 172.21.15.7:57782) 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67aca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 0x56357e9d6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54604/0 (socket says 172.21.15.7:54604) 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 -- 172.21.15.7:0/3528945811 learned_addr learned my addr 172.21.15.7:0/3528945811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f679fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 0x56357e9d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 -- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 msgr2=0x56357e9d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 0x56357e9d6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 -- 172.21.15.7:0/3528945811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 msgr2=0x56357e9d6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 0x56357e9d6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.841+0000 7f67ad400640 1 -- 172.21.15.7:0/3528945811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357e8f2950 con 0x56357e9d9670 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f679fe00640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 0x56357e9d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f67ad400640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009fd0 tx=0x7f67a0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5090793f380ba43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f6796000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0004b20 con 0x56357e9d9670 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6790004240 con 0x56357e9d9670 2024-04-20T21:41:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f67a0004cc0 con 0x56357e9d9670 2024-04-20T21:41:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67a0005020 con 0x56357e9d9670 2024-04-20T21:41:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f67b202e740 10 client.26768 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.842+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56357e8c0820 con 0x56357e9d9670 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6790004240 con 0x56357e9d9670 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f67a002f070 con 0x56357e9d9670 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 10 client.26768 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 10 client.26768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 20 client.26768 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357e9e95e0 0x56357e9eba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 4 client.26768 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.843+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357e942a70 con 0x56357e9e95e0 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.844+0000 7f67aca00640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357e9e95e0 0x56357e9eba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.844+0000 7f67aca00640 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357e9e95e0 0x56357e9eba30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5dedab9ec5875fe8 server_cookie=3487f30fc0ae2bcf in_seq=0 out_seq=0 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.844+0000 7f6796000640 10 client.26768 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f6796000640 1 -- 172.21.15.7:0/3528945811 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56357e942a70 con 0x56357e9e95e0 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f6796000640 10 client.26768 handle_command_reply: tid=0 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 client.26768 shutdown 2024-04-20T21:41:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 msgr2=0x56357e9d7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009fd0 tx=0x7f67a0004a40 comp rx=0 tx=0).stop 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 shutdown_connections 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357e9e95e0 0x56357e9eba30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d9670 0x56357e9d7080 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d0a90 0x56357e9d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 --2- 172.21.15.7:0/3528945811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0170 0x56357e9d6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f6796000640 0 client.26768 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 >> 172.21.15.7:0/3528945811 conn(0x56357e9ca8a0 msgr2=0x56357e9cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 shutdown_connections 2024-04-20T21:41:30.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.845+0000 7f67b202e740 1 -- 172.21.15.7:0/3528945811 wait complete. 2024-04-20T21:41:30.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.846+0000 7f67b202e740 20 client.26768 trim_cache size 0 max 16384 2024-04-20T21:41:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.860+0000 7f67b202e740 1 Processor -- start 2024-04-20T21:41:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.860+0000 7f67b202e740 1 -- start start 2024-04-20T21:41:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.860+0000 7f67b202e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67b202e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67b202e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67b202e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81b25c0 con 0x56357e9d0a90 2024-04-20T21:41:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67b202e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81afe80 con 0x56357e9d0170 2024-04-20T21:41:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67b202e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81adcf0 con 0x56357e9d9670 2024-04-20T21:41:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54606/0 (socket says 172.21.15.7:54606) 2024-04-20T21:41:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 -- 172.21.15.7:0/3091909757 learned_addr learned my addr 172.21.15.7:0/3091909757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67ad400640 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f67aca00640 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 -- 172.21.15.7:0/3091909757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 msgr2=0x56357e9cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 -- 172.21.15.7:0/3091909757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 msgr2=0x56357e9d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.861+0000 7f679fe00640 1 -- 172.21.15.7:0/3091909757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357e6473f0 con 0x56357e9d0a90 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.862+0000 7f679fe00640 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6790009900 tx=0x7f67900049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d6576cdcb3aff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.862+0000 7f679d200640 1 -- 172.21.15.7:0/3091909757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790042040 con 0x56357e9d0a90 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.862+0000 7f679d200640 1 -- 172.21.15.7:0/3091909757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679002cba0 con 0x56357e9d0a90 2024-04-20T21:41:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.862+0000 7f679d200640 1 -- 172.21.15.7:0/3091909757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790005230 con 0x56357e9d0a90 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- 172.21.15.7:0/3091909757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 msgr2=0x56357e9e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6790009900 tx=0x7f67900049b0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- 172.21.15.7:0/3091909757 shutdown_connections 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9e52f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 --2- 172.21.15.7:0/3091909757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- 172.21.15.7:0/3091909757 >> 172.21.15.7:0/3091909757 conn(0x56357e9ca8a0 msgr2=0x56357e9cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- 172.21.15.7:0/3091909757 shutdown_connections 2024-04-20T21:41:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- 172.21.15.7:0/3091909757 wait complete. 2024-04-20T21:41:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 -- start start 2024-04-20T21:41:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.863+0000 7f67b202e740 1 Processor -- start 2024-04-20T21:41:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81ae340 con 0x56357e9d0a90 2024-04-20T21:41:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81af7b0 con 0x56357e9d0170 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81aea10 con 0x56357e9d9670 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67b202e740 10 client.0 _ll_register_callbacks cb 0x7f67a0011a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67aca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f679fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67aca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57802/0 (socket says 172.21.15.7:57802) 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67aca00640 1 -- 172.21.15.7:0/4013938845 learned_addr learned my addr 172.21.15.7:0/4013938845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67ad400640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.864+0000 7f67aca00640 1 -- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 msgr2=0x56357e9dd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67aca00640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9dd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67aca00640 1 -- 172.21.15.7:0/4013938845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 msgr2=0x56357e9dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67aca00640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67aca00640 1 -- 172.21.15.7:0/4013938845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357e7117d0 con 0x56357e9d0170 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67ad400640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9dd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67aca00640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6798098830 tx=0x7f6798010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7579d4e54d6740a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f6796000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798020040 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f6798098a40 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679002cba0 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6798009de0 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67b202e740 10 client.26774 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56357ea4f7b0 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.865+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679002cba0 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f679809b8b0 con 0x56357e9d0170 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 10 client.26774 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 10 client.26774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 20 client.26774 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357ea4fed0 0x56357ea522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 4 client.26774 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56357e93c8e0 con 0x56357ea4fed0 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f679fe00640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357ea4fed0 0x56357ea522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f679fe00640 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357ea4fed0 0x56357ea522c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83ff9e6d75f82718 server_cookie=2764cd01f80bf449 in_seq=0 out_seq=0 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.866+0000 7f6796000640 10 client.26774 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f6796000640 1 -- 172.21.15.7:0/4013938845 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56357e93c8e0 con 0x56357ea4fed0 2024-04-20T21:41:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f6796000640 10 client.26774 handle_command_reply: tid=0 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 client.26774 shutdown 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 msgr2=0x56357e9e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6798098830 tx=0x7f6798010d70 comp rx=0 tx=0).stop 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 shutdown_connections 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56357e9d9670 0x56357e9dd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56357e9d0a90 0x56357e9dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x56357ea4fed0 0x56357ea522c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 --2- 172.21.15.7:0/4013938845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56357e9d0170 0x56357e9e19e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f6796000640 0 client.26774 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 >> 172.21.15.7:0/4013938845 conn(0x56357e9ca8a0 msgr2=0x56357e9c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 shutdown_connections 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 1 -- 172.21.15.7:0/4013938845 wait complete. 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.867+0000 7f67b202e740 20 client.26774 trim_cache size 0 max 16384 2024-04-20T21:41:30.874 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:41:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 msgr2=0x7f678c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 0x7f678c0a6260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0009600 comp rx=0 tx=0).stop 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 msgr2=0x7f67a81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f679800d950 tx=0x7f679800de10 comp rx=0 tx=0).stop 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 shutdown_connections 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f678c0a3da0 0x7f678c0a6260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a819a700 0x7f67a819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f67a81092c0 0x7f67a81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 --2- 172.21.15.7:0/3465688782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67a81041f0 0x7f67a81a0640 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 >> 172.21.15.7:0/3465688782 conn(0x7f67a80ffa50 msgr2=0x7f67a8100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 shutdown_connections 2024-04-20T21:41:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:30.868+0000 7f67ade00640 1 -- 172.21.15.7:0/3465688782 wait complete. 2024-04-20T21:41:30.882 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:30 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3950977214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:30.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:41:30.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:30 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3950977214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:30 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3950977214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:41:31.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 -- 172.21.15.7:0/1726301756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c1041c0 msgr2=0x7fe15c1045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 --2- 172.21.15.7:0/1726301756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c1041c0 0x7fe15c1045a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009900 tx=0x7fe14c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:41:31.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 -- 172.21.15.7:0/1726301756 shutdown_connections 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 --2- 172.21.15.7:0/1726301756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c104b70 0x7fe15c108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 --2- 172.21.15.7:0/1726301756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c110e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 --2- 172.21.15.7:0/1726301756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c1041c0 0x7fe15c1045a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 -- 172.21.15.7:0/1726301756 >> 172.21.15.7:0/1726301756 conn(0x7fe15c0ffa00 msgr2=0x7fe15c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 -- 172.21.15.7:0/1726301756 shutdown_connections 2024-04-20T21:41:31.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.099+0000 7fe160a00640 1 -- 172.21.15.7:0/1726301756 wait complete. 2024-04-20T21:41:31.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.100+0000 7fe160a00640 1 Processor -- start 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.100+0000 7fe160a00640 1 -- start start 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 0x7fe15c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c114390 con 0x7fe15c109290 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c114210 con 0x7fe15c1041c0 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe160a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c114510 con 0x7fe15c19a710 2024-04-20T21:41:31.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57828/0 (socket says 172.21.15.7:57828) 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15b400640 1 -- 172.21.15.7:0/4278170579 learned_addr learned my addr 172.21.15.7:0/4278170579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15aa00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15be00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 0x7fe15c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15b400640 1 -- 172.21.15.7:0/4278170579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 msgr2=0x7fe15c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.101+0000 7fe15b400640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 0x7fe15c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15b400640 1 -- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 msgr2=0x7fe15c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15b400640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15b400640 1 -- 172.21.15.7:0/4278170579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c19b310 con 0x7fe15c1041c0 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15aa00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15be00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 0x7fe15c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe15b400640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe14c02c240 tx=0x7fe14c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe153e00640 1 -- 172.21.15.7:0/4278170579 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c043020 con 0x7fe15c1041c0 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe153e00640 1 -- 172.21.15.7:0/4278170579 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c0054e0 con 0x7fe15c1041c0 2024-04-20T21:41:31.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe153e00640 1 -- 172.21.15.7:0/4278170579 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c003b50 con 0x7fe15c1041c0 2024-04-20T21:41:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.102+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c19b5a0 con 0x7fe15c1041c0 2024-04-20T21:41:31.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.103+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a76f0 con 0x7fe15c1041c0 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 Processor -- start 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe153e00640 1 -- 172.21.15.7:0/4278170579 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe14c046020 con 0x7fe15c1041c0 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 -- start start 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 0x5602186c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560218621050 con 0x5602186bc170 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602185c7490 con 0x5602186c5670 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.106+0000 7fe164c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56021861d010 con 0x5602186bca90 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 0x5602186c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 0x5602186c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-04-20T21:41:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57834/0 (socket says 172.21.15.7:57834) 2024-04-20T21:41:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15b400640 1 -- 172.21.15.7:0/2819420638 learned_addr learned my addr 172.21.15.7:0/2819420638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15be00640 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 -- 172.21.15.7:0/2819420638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 msgr2=0x5602186c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 0x5602186c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 -- 172.21.15.7:0/2819420638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 msgr2=0x5602186bc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186bc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 -- 172.21.15.7:0/2819420638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602185cf070 con 0x5602186c5670 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.107+0000 7fe15aa00640 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06b4d0 tx=0x7fe1440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9534492e62925788 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe14be00640 1 -- 172.21.15.7:0/2819420638 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144004310 con 0x5602186c5670 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe14be00640 1 -- 172.21.15.7:0/2819420638 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1440044b0 con 0x5602186c5670 2024-04-20T21:41:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe14be00640 1 -- 172.21.15.7:0/2819420638 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440035f0 con 0x5602186c5670 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe153e00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 0x7fe1400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe153e00640 1 -- 172.21.15.7:0/4278170579 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fe14c0f27f0 con 0x7fe15c1041c0 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.108+0000 7fe15be00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 0x7fe1400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- 172.21.15.7:0/2819420638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 msgr2=0x5602186c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06b4d0 tx=0x7fe1440118d0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- 172.21.15.7:0/2819420638 shutdown_connections 2024-04-20T21:41:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186c5670 0x5602186c7a60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bca90 0x5602186c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 --2- 172.21.15.7:0/2819420638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186bc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- 172.21.15.7:0/2819420638 >> 172.21.15.7:0/2819420638 conn(0x5602186b68a0 msgr2=0x5602186b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- 172.21.15.7:0/2819420638 shutdown_connections 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- 172.21.15.7:0/2819420638 wait complete. 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe15be00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 0x7fe1400a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe15400af00 tx=0x7fe15400a6b0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.109+0000 7fe164c31740 1 -- start start 2024-04-20T21:41:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 Processor -- start 2024-04-20T21:41:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5602186c8460 con 0x5602186bc170 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5602185da860 con 0x5602186bca90 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5602186c85e0 con 0x5602186c5670 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe164c31740 10 client.0 _ll_register_callbacks cb 0x5602185a8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.110+0000 7fe15b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54658/0 (socket says 172.21.15.7:54658) 2024-04-20T21:41:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 -- 172.21.15.7:0/671089023 learned_addr learned my addr 172.21.15.7:0/671089023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15be00640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15aa00640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 -- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 msgr2=0x5602186c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 -- 172.21.15.7:0/671089023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 msgr2=0x5602186c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 -- 172.21.15.7:0/671089023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602185de950 con 0x5602186bc170 2024-04-20T21:41:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe15b400640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0bb270 tx=0x7fe14c0f0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c848a32939f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe148c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0be3e0 con 0x5602186bc170 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.111+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fe14c0f01c0 con 0x5602186bc170 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1440044b0 con 0x5602186bc170 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe14c0be200 con 0x5602186bc170 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 10 client.16575 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602185ac820 con 0x5602186bc170 2024-04-20T21:41:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1440044b0 con 0x5602186bc170 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fe14c0bfbd0 con 0x5602186bc170 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 10 client.16575 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 10 client.16575 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 20 client.16575 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5602186d54d0 0x5602186d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 4 client.16575 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.112+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021862ea70 con 0x5602186d54d0 2024-04-20T21:41:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.113+0000 7fe15aa00640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5602186d54d0 0x5602186d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.113+0000 7fe15aa00640 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5602186d54d0 0x5602186d7900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b777dfa7601e753 server_cookie=7e0220b6035c601b in_seq=0 out_seq=0 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.113+0000 7fe148c00640 10 client.16575 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.114+0000 7fe148c00640 1 -- 172.21.15.7:0/671089023 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56021862ea70 con 0x5602186d54d0 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.114+0000 7fe148c00640 10 client.16575 handle_command_reply: tid=0 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.114+0000 7fe164c31740 1 client.16575 shutdown 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.114+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 msgr2=0x5602186c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.114+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0bb270 tx=0x7fe14c0f0060 comp rx=0 tx=0).stop 2024-04-20T21:41:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 shutdown_connections 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5602186d54d0 0x5602186d7900 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 --2- 172.21.15.7:0/671089023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bc170 0x5602186c2690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 >> 172.21.15.7:0/671089023 conn(0x5602186b68a0 msgr2=0x5602186b85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe148c00640 0 client.16575 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 shutdown_connections 2024-04-20T21:41:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 1 -- 172.21.15.7:0/671089023 wait complete. 2024-04-20T21:41:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.115+0000 7fe164c31740 20 client.16575 trim_cache size 0 max 16384 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 Processor -- start 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 -- start start 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 0x5602186b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1b24f0 con 0x5602186c5670 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1afdb0 con 0x5602186bca90 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe164c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1adc20 con 0x5602186bc170 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe15be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 0x5602186b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe15b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe15b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40326/0 (socket says 172.21.15.7:40326) 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.130+0000 7fe15b400640 1 -- 172.21.15.7:0/2482980325 learned_addr learned my addr 172.21.15.7:0/2482980325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15aa00640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 -- 172.21.15.7:0/2482980325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 msgr2=0x5602186d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 -- 172.21.15.7:0/2482980325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 msgr2=0x5602186b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 0x5602186b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 -- 172.21.15.7:0/2482980325 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602186bc8c0 con 0x5602186bc170 2024-04-20T21:41:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15be00640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 0x5602186b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15aa00640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:41:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe15b400640 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000c00 tx=0x7fe14c02ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50579dfb8f2cb635 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe14b400640 1 -- 172.21.15.7:0/2482980325 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0be3e0 con 0x5602186bc170 2024-04-20T21:41:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe14b400640 1 -- 172.21.15.7:0/2482980325 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14c005690 con 0x5602186bc170 2024-04-20T21:41:31.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.131+0000 7fe14b400640 1 -- 172.21.15.7:0/2482980325 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0c7510 con 0x5602186bc170 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- 172.21.15.7:0/2482980325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 msgr2=0x5602186c4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000c00 tx=0x7fe14c02ca90 comp rx=0 tx=0).stop 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- 172.21.15.7:0/2482980325 shutdown_connections 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bca90 0x5602186d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186c5670 0x5602186b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 --2- 172.21.15.7:0/2482980325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186bc170 0x5602186c4dc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- 172.21.15.7:0/2482980325 >> 172.21.15.7:0/2482980325 conn(0x5602186b68a0 msgr2=0x5602186b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- 172.21.15.7:0/2482980325 shutdown_connections 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- 172.21.15.7:0/2482980325 wait complete. 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.132+0000 7fe164c31740 1 -- start start 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 Processor -- start 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 0x5602186bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1ae270 con 0x5602186bca90 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1af6e0 con 0x5602186bc170 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c1ae940 con 0x5602186c5670 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe164c31740 10 client.0 _ll_register_callbacks cb 0x7fe14c0f8220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.133+0000 7fe15b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57860/0 (socket says 172.21.15.7:57860) 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 -- 172.21.15.7:0/1716098014 learned_addr learned my addr 172.21.15.7:0/1716098014 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15aa00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 0x5602186bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15be00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 -- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 msgr2=0x5602186bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 -- 172.21.15.7:0/1716098014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 msgr2=0x5602186bf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 0x5602186bf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 -- 172.21.15.7:0/1716098014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602183333f0 con 0x5602186bc170 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15be00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15aa00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 0x5602186bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:41:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe15b400640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe14c02c210 tx=0x7fe14c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e085c502da7ef2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe148c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0be3e0 con 0x5602186bc170 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fe14c0f0130 con 0x5602186bc170 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.134+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602186288e0 con 0x5602186bc170 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe14c0c7510 con 0x5602186bc170 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 10 client.26789 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5602183fd7d0 con 0x5602186bc170 2024-04-20T21:41:31.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5602186288e0 con 0x5602186bc170 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fe14c0bde80 con 0x5602186bc170 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 10 client.26789 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 10 client.26789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 20 client.26789 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56021873bc10 0x56021873e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 4 client.26789 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:41:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.135+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602185a45a0 con 0x56021873bc10 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.136+0000 7fe15aa00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56021873bc10 0x56021873e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.136+0000 7fe148c00640 10 client.26789 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.136+0000 7fe15aa00640 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56021873bc10 0x56021873e030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b805808c0261966b server_cookie=70e39c1f9ab0fba5 in_seq=0 out_seq=0 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe148c00640 1 -- 172.21.15.7:0/1716098014 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602185a45a0 con 0x56021873bc10 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe148c00640 10 client.26789 handle_command_reply: tid=0 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 client.26789 shutdown 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 msgr2=0x5602186bf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe14c02c210 tx=0x7fe14c0035f0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 shutdown_connections 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x56021873bc10 0x56021873e030 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5602186c5670 0x5602186bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5602186bca90 0x5602186bf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe148c00640 0 client.26789 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 --2- 172.21.15.7:0/1716098014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5602186bc170 0x5602186bf390 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 >> 172.21.15.7:0/1716098014 conn(0x5602186b68a0 msgr2=0x5602186c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 shutdown_connections 2024-04-20T21:41:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 1 -- 172.21.15.7:0/1716098014 wait complete. 2024-04-20T21:41:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.137+0000 7fe164c31740 20 client.26789 trim_cache size 0 max 16384 2024-04-20T21:41:31.146 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:41:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 msgr2=0x7fe1400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 0x7fe1400a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe15400af00 tx=0x7fe15400a6b0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 msgr2=0x7fe15c1a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe14c02c240 tx=0x7fe14c0042f0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 shutdown_connections 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe15c19a710 0x7fe15c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c109290 0x7fe15c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe1400a3d20 0x7fe1400a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 --2- 172.21.15.7:0/4278170579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe15c1041c0 0x7fe15c1a0600 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 >> 172.21.15.7:0/4278170579 conn(0x7fe15c0ffa00 msgr2=0x7fe15c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 shutdown_connections 2024-04-20T21:41:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.138+0000 7fe160a00640 1 -- 172.21.15.7:0/4278170579 wait complete. 2024-04-20T21:41:31.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:41:31.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 -- 172.21.15.7:0/27266175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc108680 msgr2=0x7f1cbc108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 --2- 172.21.15.7:0/27266175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc108680 0x7f1cbc108ac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009860 tx=0x7f1ca802bb70 comp rx=0 tx=0).stop 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 -- 172.21.15.7:0/27266175 shutdown_connections 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 --2- 172.21.15.7:0/27266175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc108680 0x7f1cbc108ac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 --2- 172.21.15.7:0/27266175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc109000 0x7f1cbc115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 --2- 172.21.15.7:0/27266175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc10a400 0x7f1cbc10a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.354+0000 7f1cc4000640 1 -- 172.21.15.7:0/27266175 >> 172.21.15.7:0/27266175 conn(0x7f1cbc077dd0 msgr2=0x7f1cbc106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.355+0000 7f1cc4000640 1 -- 172.21.15.7:0/27266175 shutdown_connections 2024-04-20T21:41:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.355+0000 7f1cc4000640 1 -- 172.21.15.7:0/27266175 wait complete. 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.356+0000 7f1cc4000640 1 Processor -- start 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.356+0000 7f1cc4000640 1 -- start start 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.356+0000 7f1cc4000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.356+0000 7f1cc4000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 0x7f1cbc19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc4000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc10a400 0x7f1cbc1a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc4000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1189d0 con 0x7f1cbc109000 2024-04-20T21:41:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc4000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc118850 con 0x7f1cbc10a400 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc4000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc118b50 con 0x7f1cbc108680 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 0x7f1cbc19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40356/0 (socket says 172.21.15.7:40356) 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 -- 172.21.15.7:0/2411692805 learned_addr learned my addr 172.21.15.7:0/2411692805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc10a400 0x7f1cbc1a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 -- 172.21.15.7:0/2411692805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc10a400 msgr2=0x7f1cbc1a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc10a400 0x7f1cbc1a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 -- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 msgr2=0x7f1cbc19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 0x7f1cbc19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc1600640 1 -- 172.21.15.7:0/2411692805 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc1a3e20 con 0x7f1cbc108680 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.357+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 0x7f1cbc19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9c0 tx=0x7f1cb000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cba000640 1 -- 172.21.15.7:0/2411692805 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000cde0 con 0x7f1cbc108680 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc1a4110 con 0x7f1cbc108680 2024-04-20T21:41:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cba000640 1 -- 172.21.15.7:0/2411692805 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb00035f0 con 0x7f1cbc108680 2024-04-20T21:41:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cba000640 1 -- 172.21.15.7:0/2411692805 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0010650 con 0x7f1cbc108680 2024-04-20T21:41:31.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.358+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc1ab9a0 con 0x7f1cbc108680 2024-04-20T21:41:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.361+0000 7f1cc6db8740 1 Processor -- start 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 -- start start 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cba000640 1 -- 172.21.15.7:0/2411692805 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1cb00037a0 con 0x7f1cbc108680 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 0x559822a0cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a0d4b0 0x559822a0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559822969050 con 0x559822a03fb0 2024-04-20T21:41:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55982290f490 con 0x559822a0d4b0 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559822965010 con 0x559822a048d0 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc1600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 0x559822a0cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54712/0 (socket says 172.21.15.7:54712) 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc2000640 1 -- 172.21.15.7:0/2629012737 learned_addr learned my addr 172.21.15.7:0/2629012737 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.362+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a0d4b0 0x559822a0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 -- 172.21.15.7:0/2629012737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 msgr2=0x559822a0cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 0x559822a0cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 -- 172.21.15.7:0/2629012737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a0d4b0 msgr2=0x559822a0f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a0d4b0 0x559822a0f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 -- 172.21.15.7:0/2629012737 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559822917070 con 0x559822a03fb0 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 0x559822a0cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fc0 tx=0x7f1cb40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4965928231563176 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.363+0000 7f1cae600640 1 -- 172.21.15.7:0/2629012737 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4004d20 con 0x559822a03fb0 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cae600640 1 -- 172.21.15.7:0/2629012737 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4004050 con 0x559822a03fb0 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cae600640 1 -- 172.21.15.7:0/2629012737 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4005260 con 0x559822a03fb0 2024-04-20T21:41:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cba000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cba000640 1 -- 172.21.15.7:0/2411692805 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1cb00d0b40 con 0x7f1cbc108680 2024-04-20T21:41:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2629012737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 msgr2=0x559822a04390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009fc0 tx=0x7f1cb40118d0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2629012737 shutdown_connections 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a0d4b0 0x559822a0f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a048d0 0x559822a0cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2629012737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a03fb0 0x559822a04390 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.364+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2629012737 >> 172.21.15.7:0/2629012737 conn(0x5598229fe6e0 msgr2=0x5598229feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.365+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2629012737 shutdown_connections 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.365+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2629012737 wait complete. 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.365+0000 7f1cc1600640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559822a03a80 tx=0x7f1cb00d45f0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.365+0000 7f1cc6db8740 1 -- start start 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.365+0000 7f1cc6db8740 1 Processor -- start 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a0afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559822a102a0 con 0x559822a0d4b0 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559822922860 con 0x559822a048d0 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x559822a10420 con 0x559822a03fb0 2024-04-20T21:41:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc6db8740 10 client.0 _ll_register_callbacks cb 0x5598228f0fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a0afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57910/0 (socket says 172.21.15.7:57910) 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.366+0000 7f1cc0c00640 1 -- 172.21.15.7:0/2193412398 learned_addr learned my addr 172.21.15.7:0/2193412398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 -- 172.21.15.7:0/2193412398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 msgr2=0x559822a0a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a0a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 -- 172.21.15.7:0/2193412398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 msgr2=0x559822a0afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a0afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 -- 172.21.15.7:0/2193412398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559822926950 con 0x559822a048d0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8000c00 tx=0x7f1ca8004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53ccef24979e2a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1ca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8039040 con 0x559822a048d0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1ca8034640 con 0x559822a048d0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.367+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb4004050 con 0x559822a048d0 2024-04-20T21:41:31.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ca8050050 con 0x559822a048d0 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 10 client.26801 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598228f4820 con 0x559822a048d0 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4004050 con 0x559822a048d0 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1ca8047070 con 0x559822a048d0 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 10 client.26801 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 20 client.26801 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a1d500 0x559822a1f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 4 client.26801 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.368+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559822976a70 con 0x559822a1d500 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.369+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a1d500 0x559822a1f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.369+0000 7f1cc2000640 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a1d500 0x559822a1f920 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8549055f64dd7eb8 server_cookie=69d4b1daee802f1b in_seq=0 out_seq=0 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.369+0000 7f1ca2a00640 10 client.26801 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1ca2a00640 1 -- 172.21.15.7:0/2193412398 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559822976a70 con 0x559822a1d500 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1ca2a00640 10 client.26801 handle_command_reply: tid=0 2024-04-20T21:41:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 client.26801 shutdown 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 msgr2=0x559822a0aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8000c00 tx=0x7f1ca8004580 comp rx=0 tx=0).stop 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 shutdown_connections 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0aa90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a1d500 0x559822a1f920 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a0afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/2193412398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a0a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1ca2a00640 0 client.26801 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.370+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 >> 172.21.15.7:0/2193412398 conn(0x5598229fe6e0 msgr2=0x559822a00430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.371+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 shutdown_connections 2024-04-20T21:41:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.371+0000 7f1cc6db8740 1 -- 172.21.15.7:0/2193412398 wait complete. 2024-04-20T21:41:31.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.371+0000 7f1cc6db8740 20 client.26801 trim_cache size 0 max 16384 2024-04-20T21:41:31.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 Processor -- start 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 -- start start 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a19160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b6a60 con 0x559822a0d4b0 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b4320 con 0x559822a048d0 2024-04-20T21:41:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.386+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b2190 con 0x559822a03fb0 2024-04-20T21:41:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a19160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a19160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40372/0 (socket says 172.21.15.7:40372) 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc2000640 1 -- 172.21.15.7:0/351625802 learned_addr learned my addr 172.21.15.7:0/351625802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc2000640 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 -- 172.21.15.7:0/351625802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 msgr2=0x559822a19160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a19160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 -- 172.21.15.7:0/351625802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 msgr2=0x559822a251d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a251d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 -- 172.21.15.7:0/351625802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559822a04700 con 0x559822a048d0 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc2000640 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0adf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8032a60 tx=0x7f1ca802bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca4bbd1f3bc15eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cadc00640 1 -- 172.21.15.7:0/351625802 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca804d020 con 0x559822a048d0 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cadc00640 1 -- 172.21.15.7:0/351625802 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8005850 con 0x559822a048d0 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.387+0000 7f1cadc00640 1 -- 172.21.15.7:0/351625802 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8039070 con 0x559822a048d0 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 -- 172.21.15.7:0/351625802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 msgr2=0x559822a0adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0adf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8032a60 tx=0x7f1ca802bb70 comp rx=0 tx=0).stop 2024-04-20T21:41:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 -- 172.21.15.7:0/351625802 shutdown_connections 2024-04-20T21:41:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a048d0 0x559822a0adf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a0d4b0 0x559822a251d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/351625802 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a03fb0 0x559822a19160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 -- 172.21.15.7:0/351625802 >> 172.21.15.7:0/351625802 conn(0x5598229fe6e0 msgr2=0x5598229fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 -- 172.21.15.7:0/351625802 shutdown_connections 2024-04-20T21:41:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.388+0000 7f1cc6db8740 1 -- 172.21.15.7:0/351625802 wait complete. 2024-04-20T21:41:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.389+0000 7f1cc6db8740 1 -- start start 2024-04-20T21:41:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.389+0000 7f1cc6db8740 1 Processor -- start 2024-04-20T21:41:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.389+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a03fb0 0x559822a11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.389+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a048d0 0x559822a11690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc6db8740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b27e0 con 0x559822a048d0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b3c50 con 0x559822a03fb0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc6db8740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc1b2eb0 con 0x559822a0d4b0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc6db8740 10 client.0 _ll_register_callbacks cb 0x7f1ca8004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a03fb0 0x559822a11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a048d0 0x559822a11690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40374/0 (socket says 172.21.15.7:40374) 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 -- 172.21.15.7:0/3157965384 learned_addr learned my addr 172.21.15.7:0/3157965384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 -- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a03fb0 msgr2=0x559822a11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a03fb0 0x559822a11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 -- 172.21.15.7:0/3157965384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a048d0 msgr2=0x559822a11690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a048d0 0x559822a11690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.390+0000 7f1cc2000640 1 -- 172.21.15.7:0/3157965384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982267b3f0 con 0x559822a0d4b0 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1cc2000640 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009db0 tx=0x7f1cb4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563112540692adf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:31.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1ca2a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4003b10 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca8005850 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1cb4003cb0 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cb400f8c0 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1cc6db8740 10 client.26764 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.391+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598227457d0 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8005850 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1cb4025040 con 0x559822a0d4b0 2024-04-20T21:41:31.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 10 client.26764 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 10 client.26764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 20 client.26764 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a840d0 0x559822a864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 4 client.26764 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.392+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598229708e0 con 0x559822a840d0 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a840d0 0x559822a864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1cc0c00640 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a840d0 0x559822a864f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9065696b22449996 server_cookie=e2197920e45c5ef3 in_seq=0 out_seq=0 2024-04-20T21:41:31.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1ca2a00640 10 client.26764 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1ca2a00640 1 -- 172.21.15.7:0/3157965384 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598229708e0 con 0x559822a840d0 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1ca2a00640 10 client.26764 handle_command_reply: tid=0 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.393+0000 7f1cc6db8740 1 client.26764 shutdown 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 msgr2=0x559822a11bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009db0 tx=0x7f1cb4005fd0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 shutdown_connections 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x559822a840d0 0x559822a864f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559822a0d4b0 0x559822a11bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559822a048d0 0x559822a11690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1ca2a00640 0 client.26764 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 --2- 172.21.15.7:0/3157965384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559822a03fb0 0x559822a11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 >> 172.21.15.7:0/3157965384 conn(0x5598229fe6e0 msgr2=0x5598229ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 shutdown_connections 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 1 -- 172.21.15.7:0/3157965384 wait complete. 2024-04-20T21:41:31.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.394+0000 7f1cc6db8740 20 client.26764 trim_cache size 0 max 16384 2024-04-20T21:41:31.401 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:41:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 msgr2=0x7f1ca40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x559822a03a80 tx=0x7f1cb00d45f0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 msgr2=0x7f1cbc19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e9c0 tx=0x7f1cb000ee80 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 shutdown_connections 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1ca40a3d20 0x7f1ca40a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cbc10a400 0x7f1cbc1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc109000 0x7f1cbc19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 --2- 172.21.15.7:0/2411692805 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1cbc108680 0x7f1cbc19ee30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 >> 172.21.15.7:0/2411692805 conn(0x7f1cbc077dd0 msgr2=0x7f1cbc078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 shutdown_connections 2024-04-20T21:41:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:31.395+0000 7f1cc4000640 1 -- 172.21.15.7:0/2411692805 wait complete. 2024-04-20T21:41:31.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:31 smithi016 ceph-mon[21409]: pgmap v253: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:31.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:31 smithi186 ceph-mon[22157]: pgmap v253: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:32.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:31 smithi007 ceph-mon[16867]: pgmap v253: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:32.412 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:41:32.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:41:32.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:41:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 -- 172.21.15.7:0/1528806147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb1010c390 msgr2=0x7fcb1010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 --2- 172.21.15.7:0/1528806147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb1010c390 0x7fcb1010e7e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009be0 tx=0x7fcb0802bef0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 -- 172.21.15.7:0/1528806147 shutdown_connections 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 --2- 172.21.15.7:0/1528806147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 0x7fcb1010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 --2- 172.21.15.7:0/1528806147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb1010c390 0x7fcb1010e7e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 --2- 172.21.15.7:0/1528806147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb10102f20 0x7fcb10103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 -- 172.21.15.7:0/1528806147 >> 172.21.15.7:0/1528806147 conn(0x7fcb100fc7a0 msgr2=0x7fcb100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 -- 172.21.15.7:0/1528806147 shutdown_connections 2024-04-20T21:41:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.644+0000 7fcb15000640 1 -- 172.21.15.7:0/1528806147 wait complete. 2024-04-20T21:41:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.645+0000 7fcb15000640 1 Processor -- start 2024-04-20T21:41:32.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 -- start start 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 0x7fcb101a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 0x7fcb101a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10112110 con 0x7fcb1010c390 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111f90 con 0x7fcb10103840 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb15000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10112290 con 0x7fcb10102f20 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 0x7fcb101a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54770/0 (socket says 172.21.15.7:54770) 2024-04-20T21:41:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 0x7fcb101a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40396/0 (socket says 172.21.15.7:40396) 2024-04-20T21:41:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.646+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3793469071 learned_addr learned my addr 172.21.15.7:0/3793469071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0ea00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 0x7fcb101a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3793469071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 msgr2=0x7fcb101a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 0x7fcb101a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3793469071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 msgr2=0x7fcb101a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 0x7fcb101a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3793469071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019af20 con 0x7fcb1010c390 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.647+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcb080096f0 tx=0x7fcb080336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.648+0000 7fcb07e00640 1 -- 172.21.15.7:0/3793469071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08043020 con 0x7fcb1010c390 2024-04-20T21:41:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.648+0000 7fcb07e00640 1 -- 172.21.15.7:0/3793469071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb080338a0 con 0x7fcb1010c390 2024-04-20T21:41:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.648+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019b1b0 con 0x7fcb1010c390 2024-04-20T21:41:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.648+0000 7fcb07e00640 1 -- 172.21.15.7:0/3793469071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0803e6b0 con 0x7fcb1010c390 2024-04-20T21:41:32.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.648+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a7520 con 0x7fcb1010c390 2024-04-20T21:41:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.651+0000 7fcb19368740 1 Processor -- start 2024-04-20T21:41:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.651+0000 7fcb07e00640 1 -- 172.21.15.7:0/3793469071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcb08046020 con 0x7fcb1010c390 2024-04-20T21:41:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.651+0000 7fcb19368740 1 -- start start 2024-04-20T21:41:32.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56374edd4d20 con 0x56374ee700f0 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56374ed7b3e0 con 0x56374ee795f0 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb19368740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56374edd0ce0 con 0x56374ee70a10 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:40410/0 (socket says 172.21.15.7:40410) 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 -- 172.21.15.7:0/1172753134 learned_addr learned my addr 172.21.15.7:0/1172753134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0ea00640 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 -- 172.21.15.7:0/1172753134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 msgr2=0x56374ee7b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 -- 172.21.15.7:0/1172753134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 msgr2=0x56374ee704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.652+0000 7fcb0f400640 1 -- 172.21.15.7:0/1172753134 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374ed9a980 con 0x56374ee70a10 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.653+0000 7fcb0f400640 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009cd0 tx=0x7fcb00011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5808a8b5e0d7479 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.653+0000 7fcb04c00640 1 -- 172.21.15.7:0/1172753134 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001e070 con 0x56374ee70a10 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.653+0000 7fcb04c00640 1 -- 172.21.15.7:0/1172753134 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb00019b10 con 0x56374ee70a10 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.653+0000 7fcb04c00640 1 -- 172.21.15.7:0/1172753134 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000eb50 con 0x56374ee70a10 2024-04-20T21:41:32.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.653+0000 7fcb07e00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 0x7fcaf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb07e00640 1 -- 172.21.15.7:0/3793469071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fcb080f2c80 con 0x7fcb1010c390 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb0ea00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 0x7fcaf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 -- 172.21.15.7:0/1172753134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 msgr2=0x56374ee790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009cd0 tx=0x7fcb00011810 comp rx=0 tx=0).stop 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 -- 172.21.15.7:0/1172753134 shutdown_connections 2024-04-20T21:41:32.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee790b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 --2- 172.21.15.7:0/1172753134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 -- 172.21.15.7:0/1172753134 >> 172.21.15.7:0/1172753134 conn(0x56374ee6a820 msgr2=0x56374ee6ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 -- 172.21.15.7:0/1172753134 shutdown_connections 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb19368740 1 -- 172.21.15.7:0/1172753134 wait complete. 2024-04-20T21:41:32.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.654+0000 7fcb0ea00640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 0x7fcaf40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56374ee6b9b0 tx=0x7fcafc009770 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 -- start start 2024-04-20T21:41:32.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 Processor -- start 2024-04-20T21:41:32.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee70a10 0x56374ee76b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 0x56374ee770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56374ee7c3e0 con 0x56374ee70a10 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56374ed8e7a0 con 0x56374ee700f0 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.655+0000 7fcb19368740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56374ee7c560 con 0x56374ee795f0 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb19368740 10 client.0 _ll_register_callbacks cb 0x56374ed90b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57968/0 (socket says 172.21.15.7:57968) 2024-04-20T21:41:32.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0f400640 1 -- 172.21.15.7:0/868633544 learned_addr learned my addr 172.21.15.7:0/868633544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0ea00640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee70a10 0x56374ee76b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 0x56374ee770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 -- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 msgr2=0x56374ee76620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 -- 172.21.15.7:0/868633544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee70a10 msgr2=0x56374ee76b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee70a10 0x56374ee76b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0fe00640 1 -- 172.21.15.7:0/868633544 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374ebaf7e0 con 0x56374ee795f0 2024-04-20T21:41:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.656+0000 7fcb0f400640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 0x56374ee770a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcb080f04a0 tx=0x7fcb080be960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110b2ca225121ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080d8060 con 0x56374ee795f0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb00019b10 con 0x56374ee795f0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fcb08003c10 con 0x56374ee795f0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb08004820 con 0x56374ee795f0 2024-04-20T21:41:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcb19368740 10 client.26776 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56374ec0ff70 con 0x56374ee795f0 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.657+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb00019b10 con 0x56374ee795f0 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fcb080df140 con 0x56374ee795f0 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 10 client.26776 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 20 client.26776 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:32.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374ee89940 0x56374ee8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 4 client.26776 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374ede2740 con 0x56374ee89940 2024-04-20T21:41:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.658+0000 7fcb0f400640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374ee89940 0x56374ee8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.659+0000 7fcb0f400640 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374ee89940 0x56374ee8bde0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e071b843124dd7 server_cookie=89fec5cc42deea0f in_seq=0 out_seq=0 2024-04-20T21:41:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.659+0000 7fcaf9600640 10 client.26776 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.659+0000 7fcaf9600640 1 -- 172.21.15.7:0/868633544 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56374ede2740 con 0x56374ee89940 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.659+0000 7fcaf9600640 10 client.26776 handle_command_reply: tid=0 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.659+0000 7fcb19368740 1 client.26776 shutdown 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 msgr2=0x56374ee770a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 0x56374ee770a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcb080f04a0 tx=0x7fcb080be960 comp rx=0 tx=0).stop 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 shutdown_connections 2024-04-20T21:41:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374ee89940 0x56374ee8bde0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee795f0 0x56374ee770a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee70a10 0x56374ee76b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcaf9600640 0 client.26776 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 --2- 172.21.15.7:0/868633544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee76620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 >> 172.21.15.7:0/868633544 conn(0x56374ee6a820 msgr2=0x56374ee6c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 shutdown_connections 2024-04-20T21:41:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 1 -- 172.21.15.7:0/868633544 wait complete. 2024-04-20T21:41:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.660+0000 7fcb19368740 20 client.26776 trim_cache size 0 max 16384 2024-04-20T21:41:32.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.679+0000 7fcb19368740 1 Processor -- start 2024-04-20T21:41:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 -- start start 2024-04-20T21:41:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 0x56374ee69880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b25d0 con 0x56374ee795f0 2024-04-20T21:41:32.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101afe90 con 0x56374ee700f0 2024-04-20T21:41:32.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb19368740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101add00 con 0x56374ee70a10 2024-04-20T21:41:32.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb0ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57978/0 (socket says 172.21.15.7:57978) 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb0f400640 1 -- 172.21.15.7:0/511235949 learned_addr learned my addr 172.21.15.7:0/511235949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.680+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 0x56374ee69880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 -- 172.21.15.7:0/511235949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 msgr2=0x56374ee852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee852d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 -- 172.21.15.7:0/511235949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 msgr2=0x56374ee69880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 0x56374ee69880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 -- 172.21.15.7:0/511235949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374eaa7170 con 0x56374ee700f0 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 0x56374ee69880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcb0f400640 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e720 tx=0x7fcb0000e610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30a57dc8bf706ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcafbe00640 1 -- 172.21.15.7:0/511235949 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001e040 con 0x56374ee700f0 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcafbe00640 1 -- 172.21.15.7:0/511235949 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0001f040 con 0x56374ee700f0 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.681+0000 7fcafbe00640 1 -- 172.21.15.7:0/511235949 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0002d7a0 con 0x56374ee700f0 2024-04-20T21:41:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- 172.21.15.7:0/511235949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 msgr2=0x56374ee78d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000e720 tx=0x7fcb0000e610 comp rx=0 tx=0).stop 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- 172.21.15.7:0/511235949 shutdown_connections 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee795f0 0x56374ee69880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 --2- 172.21.15.7:0/511235949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee700f0 0x56374ee78d50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- 172.21.15.7:0/511235949 >> 172.21.15.7:0/511235949 conn(0x56374ee6a820 msgr2=0x56374ee69170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- 172.21.15.7:0/511235949 shutdown_connections 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- 172.21.15.7:0/511235949 wait complete. 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.682+0000 7fcb19368740 1 -- start start 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 Processor -- start 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101ae350 con 0x56374ee700f0 2024-04-20T21:41:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101af7c0 con 0x56374ee795f0 2024-04-20T21:41:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101aea20 con 0x56374ee70a10 2024-04-20T21:41:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb19368740 10 client.0 _ll_register_callbacks cb 0x7fcb08009280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57992/0 (socket says 172.21.15.7:57992) 2024-04-20T21:41:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb0ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3018879588 learned_addr learned my addr 172.21.15.7:0/3018879588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.683+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3018879588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 msgr2=0x56374ee7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee7cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3018879588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 msgr2=0x56374ee81c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee81c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 -- 172.21.15.7:0/3018879588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374ee7dd50 con 0x56374ee795f0 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb0fe00640 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcb0804fff0 tx=0x7fcb08004d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa479b59dd34b1df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcaf9600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:41:32.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080d8270 con 0x56374ee795f0 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fcb080df2b0 con 0x56374ee795f0 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0001f040 con 0x56374ee795f0 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcb080becf0 con 0x56374ee795f0 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.684+0000 7fcb19368740 10 client.26825 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56374ee7dd50 con 0x56374ee795f0 2024-04-20T21:41:32.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0001f040 con 0x56374ee795f0 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fcb080f0ae0 con 0x56374ee795f0 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 10 client.26825 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 10 client.26825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 20 client.26825 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374eef0380 0x56374eef2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 4 client.26825 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:41:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56374eddc5b0 con 0x56374eef0380 2024-04-20T21:41:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.685+0000 7fcb0f400640 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374eef0380 0x56374eef2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.686+0000 7fcb0f400640 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374eef0380 0x56374eef2770 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6e63a80aeb8a31 server_cookie=576afd6d049dec2 in_seq=0 out_seq=0 2024-04-20T21:41:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.686+0000 7fcaf9600640 10 client.26825 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcaf9600640 1 -- 172.21.15.7:0/3018879588 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56374eddc5b0 con 0x56374eef0380 2024-04-20T21:41:32.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcaf9600640 10 client.26825 handle_command_reply: tid=0 2024-04-20T21:41:32.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 client.26825 shutdown 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 msgr2=0x56374ee7d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcb0804fff0 tx=0x7fcb08004d70 comp rx=0 tx=0).stop 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 shutdown_connections 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56374ee795f0 0x56374ee7d4d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56374eef0380 0x56374eef2770 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56374ee70a10 0x56374ee7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 --2- 172.21.15.7:0/3018879588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56374ee700f0 0x56374ee81c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 >> 172.21.15.7:0/3018879588 conn(0x56374ee6a820 msgr2=0x56374ee6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcaf9600640 0 client.26825 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.687+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 shutdown_connections 2024-04-20T21:41:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb19368740 1 -- 172.21.15.7:0/3018879588 wait complete. 2024-04-20T21:41:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb19368740 20 client.26825 trim_cache size 0 max 16384 2024-04-20T21:41:32.697 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:41:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:41:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d1b19819-c8cf-4e31-b92c-4bd3c7f0ef21", 2024-04-20T21:41:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:41:32.698 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:41:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 msgr2=0x7fcaf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 0x7fcaf40a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56374ee6b9b0 tx=0x7fcafc009770 comp rx=0 tx=0).stop 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 msgr2=0x7fcb1019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcb080096f0 tx=0x7fcb080336b0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.688+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 shutdown_connections 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fcaf40a3d20 0x7fcaf40a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb10103840 0x7fcb101a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb1010c390 0x7fcb1019a660 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 --2- 172.21.15.7:0/3793469071 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb10102f20 0x7fcb101a05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 >> 172.21.15.7:0/3793469071 conn(0x7fcb100fc7a0 msgr2=0x7fcb10100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 shutdown_connections 2024-04-20T21:41:32.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.689+0000 7fcb15000640 1 -- 172.21.15.7:0/3793469071 wait complete. 2024-04-20T21:41:32.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:41:32.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:41:32.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 -- 172.21.15.7:0/2584949311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 msgr2=0x7f9a98114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/2584949311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a98114fe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009ca0 tx=0x7f9a9002c100 comp rx=0 tx=0).stop 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 -- 172.21.15.7:0/2584949311 shutdown_connections 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/2584949311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a98114fe0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/2584949311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/2584949311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a98076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.913+0000 7f9a9ca00640 1 -- 172.21.15.7:0/2584949311 >> 172.21.15.7:0/2584949311 conn(0x7f9a980ffa00 msgr2=0x7f9a98101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.914+0000 7f9a9ca00640 1 -- 172.21.15.7:0/2584949311 shutdown_connections 2024-04-20T21:41:32.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.914+0000 7f9a9ca00640 1 -- 172.21.15.7:0/2584949311 wait complete. 2024-04-20T21:41:32.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 Processor -- start 2024-04-20T21:41:32.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 -- start start 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a981a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a9819eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98118730 con 0x7f9a98112bf0 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981185b0 con 0x7f9a98076490 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a9ca00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981188b0 con 0x7f9a98076db0 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a96800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a981a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:47500/0 (socket says 172.21.15.7:47500) 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.915+0000 7f9a8fe00640 1 -- 172.21.15.7:0/3568912347 learned_addr learned my addr 172.21.15.7:0/3568912347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a97200640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a9819eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 -- 172.21.15.7:0/3568912347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 msgr2=0x7f9a981a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 -- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 msgr2=0x7f9a9819eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a9819eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 -- 172.21.15.7:0/3568912347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9819f230 con 0x7f9a98076490 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.916+0000 7f9a96800640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a981a48b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9f0 tx=0x7f9a8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.917+0000 7f9a8f400640 1 -- 172.21.15.7:0/3568912347 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800ccc0 con 0x7f9a98076490 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.917+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9819f520 con 0x7f9a98076490 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.917+0000 7f9a8f400640 1 -- 172.21.15.7:0/3568912347 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88010910 con 0x7f9a98076490 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.917+0000 7f9a8f400640 1 -- 172.21.15.7:0/3568912347 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88018960 con 0x7f9a98076490 2024-04-20T21:41:32.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.917+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981ab770 con 0x7f9a98076490 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 Processor -- start 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 -- start start 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9a8f400640 1 -- 172.21.15.7:0/3568912347 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9a88010ac0 con 0x7f9a98076490 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf71927760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf71930340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf71932c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf7188d150 con 0x55bf71927380 2024-04-20T21:41:32.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf71833370 con 0x55bf71930880 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.920+0000 7f9aa017a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf71889110 con 0x55bf71927ca0 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf71927760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf71932c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf71932c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37696/0 (socket says 172.21.15.7:37696) 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a8fe00640 1 -- 172.21.15.7:0/1813061112 learned_addr learned my addr 172.21.15.7:0/1813061112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a96800640 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf71930340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 -- 172.21.15.7:0/1813061112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 msgr2=0x55bf71930340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf71930340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 -- 172.21.15.7:0/1813061112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 msgr2=0x55bf71932c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf71932c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.921+0000 7f9a97200640 1 -- 172.21.15.7:0/1813061112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf716697d0 con 0x55bf71927380 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a97200640 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf71927760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a9002c640 tx=0x7f9a90009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d69464bff68ee71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a8f400640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 0x7f9a7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a87400640 1 -- 172.21.15.7:0/1813061112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90045020 con 0x55bf71927380 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a87400640 1 -- 172.21.15.7:0/1813061112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a9003a040 con 0x55bf71927380 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a87400640 1 -- 172.21.15.7:0/1813061112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90039040 con 0x55bf71927380 2024-04-20T21:41:32.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a8f400640 1 -- 172.21.15.7:0/3568912347 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f9a880d0b70 con 0x7f9a98076490 2024-04-20T21:41:32.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.922+0000 7f9a96800640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 0x7f9a7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 -- 172.21.15.7:0/1813061112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 msgr2=0x55bf71927760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf71927760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a9002c640 tx=0x7f9a90009900 comp rx=0 tx=0).stop 2024-04-20T21:41:32.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9a96800640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 0x7f9a7c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55bf71926e50 tx=0x7f9a88009700 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 -- 172.21.15.7:0/1813061112 shutdown_connections 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf71932c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf71930340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 --2- 172.21.15.7:0/1813061112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf71927760 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 -- 172.21.15.7:0/1813061112 >> 172.21.15.7:0/1813061112 conn(0x55bf71921ab0 msgr2=0x55bf71921e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 -- 172.21.15.7:0/1813061112 shutdown_connections 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.923+0000 7f9aa017a740 1 -- 172.21.15.7:0/1813061112 wait complete. 2024-04-20T21:41:32.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 -- start start 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 Processor -- start 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf7192de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf7192e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.924+0000 7f9aa017a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55bf71933610 con 0x55bf71927380 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9aa017a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bf71846740 con 0x55bf71930880 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9aa017a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55bf71933790 con 0x55bf71927ca0 2024-04-20T21:41:32.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9aa017a740 10 client.0 _ll_register_callbacks cb 0x55bf71841b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38710/0 (socket says 172.21.15.7:38710) 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 -- 172.21.15.7:0/3060656533 learned_addr learned my addr 172.21.15.7:0/3060656533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf7192de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf7192e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 -- 172.21.15.7:0/3060656533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 msgr2=0x55bf7192de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf7192de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 -- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 msgr2=0x55bf7192e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf7192e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.925+0000 7f9a97200640 1 -- 172.21.15.7:0/3060656533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf7159f3f0 con 0x55bf71927380 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9a97200640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009ca0 tx=0x7f9a900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e9a9a1935b72eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9a7be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90045070 con 0x55bf71927380 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f9a900327a0 con 0x55bf71927380 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a9003a040 con 0x55bf71927380 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a9005d050 con 0x55bf71927380 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9aa017a740 10 client.16626 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:32.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.926+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf7155f0f0 con 0x55bf71927380 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a9003a040 con 0x55bf71927380 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9a90041430 con 0x55bf71927380 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 10 client.16626 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 20 client.16626 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf71940ac0 0x55bf71942eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 4 client.16626 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf7189ab70 con 0x55bf71940ac0 2024-04-20T21:41:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.927+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf71940ac0 0x55bf71942eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.928+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf71940ac0 0x55bf71942eb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7da288127f4f62 server_cookie=250828dbe144cb3b in_seq=0 out_seq=0 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.928+0000 7f9a7be00640 10 client.16626 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9a7be00640 1 -- 172.21.15.7:0/3060656533 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf7189ab70 con 0x55bf71940ac0 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9a7be00640 10 client.16626 handle_command_reply: tid=0 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 client.16626 shutdown 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 msgr2=0x55bf7192d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009ca0 tx=0x7f9a900095c0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 shutdown_connections 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf7192e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf71940ac0 0x55bf71942eb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927ca0 0x55bf7192de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 --2- 172.21.15.7:0/3060656533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927380 0x55bf7192d950 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9a7be00640 0 client.16626 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 >> 172.21.15.7:0/3060656533 conn(0x55bf71921ab0 msgr2=0x55bf71923800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 shutdown_connections 2024-04-20T21:41:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 1 -- 172.21.15.7:0/3060656533 wait complete. 2024-04-20T21:41:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.929+0000 7f9aa017a740 20 client.16626 trim_cache size 0 max 16384 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 Processor -- start 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 -- start start 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71930880 0x55bf71945ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b6880 con 0x55bf71927ca0 2024-04-20T21:41:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b4140 con 0x55bf71927380 2024-04-20T21:41:32.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.949+0000 7f9aa017a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b1fb0 con 0x55bf71930880 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a97200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38716/0 (socket says 172.21.15.7:38716) 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 -- 172.21.15.7:0/407474555 learned_addr learned my addr 172.21.15.7:0/407474555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a97200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37702/0 (socket says 172.21.15.7:37702) 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a97200640 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71930880 0x55bf71945ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 -- 172.21.15.7:0/407474555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71930880 msgr2=0x55bf71945ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71930880 0x55bf71945ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 -- 172.21.15.7:0/407474555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 msgr2=0x55bf71930080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 -- 172.21.15.7:0/407474555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf71926470 con 0x55bf71927ca0 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a97200640 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.950+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005d00 tx=0x7f9a80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332ed239411ce8ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9a86a00640 1 -- 172.21.15.7:0/407474555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80005020 con 0x55bf71927ca0 2024-04-20T21:41:32.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9a86a00640 1 -- 172.21.15.7:0/407474555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8000f810 con 0x55bf71927ca0 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9a86a00640 1 -- 172.21.15.7:0/407474555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80004a90 con 0x55bf71927ca0 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 -- 172.21.15.7:0/407474555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 msgr2=0x55bf71920990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a80005d00 tx=0x7f9a80005f90 comp rx=0 tx=0).stop 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 -- 172.21.15.7:0/407474555 shutdown_connections 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71930880 0x55bf71945ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71920990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 --2- 172.21.15.7:0/407474555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71927380 0x55bf71930080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.951+0000 7f9aa017a740 1 -- 172.21.15.7:0/407474555 >> 172.21.15.7:0/407474555 conn(0x55bf71921ab0 msgr2=0x55bf71920280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.952+0000 7f9aa017a740 1 -- 172.21.15.7:0/407474555 shutdown_connections 2024-04-20T21:41:32.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.952+0000 7f9aa017a740 1 -- 172.21.15.7:0/407474555 wait complete. 2024-04-20T21:41:32.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.952+0000 7f9aa017a740 1 -- start start 2024-04-20T21:41:32.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.952+0000 7f9aa017a740 1 Processor -- start 2024-04-20T21:41:32.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927380 0x55bf71943b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf719445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b2600 con 0x55bf71927ca0 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b3a70 con 0x55bf71930880 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b2cd0 con 0x55bf71927380 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9aa017a740 10 client.0 _ll_register_callbacks cb 0x7f9a80009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38720/0 (socket says 172.21.15.7:38720) 2024-04-20T21:41:32.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 -- 172.21.15.7:0/4092764840 learned_addr learned my addr 172.21.15.7:0/4092764840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a97200640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927380 0x55bf71943b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a97200640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf719445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 -- 172.21.15.7:0/4092764840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927380 msgr2=0x55bf71943b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927380 0x55bf71943b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 -- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 msgr2=0x55bf719445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf719445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.953+0000 7f9a8fe00640 1 -- 172.21.15.7:0/4092764840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf719a5ea0 con 0x55bf71927ca0 2024-04-20T21:41:32.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a8fe00640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000c00 tx=0x7f9a80012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec2f89129b2d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a7be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8002f070 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f9a8000fdd0 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8000f810 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9a8003b050 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9aa017a740 10 client.16632 fetch_fsmap learned FSMap version 27 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf719a7170 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.954+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8000f810 con 0x55bf71927ca0 2024-04-20T21:41:32.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9a8002b8d0 con 0x55bf71927ca0 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 10 client.16632 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 20 client.16632 populate_metadata read hostname 'smithi007' 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf719a7810 0x55bf719a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 4 client.16632 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf718949e0 con 0x55bf719a7810 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.955+0000 7f9a97200640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf719a7810 0x55bf719a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:41:32.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9a97200640 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf719a7810 0x55bf719a9c00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162f33d9912da1c7 server_cookie=e0f98b333a610ea5 in_seq=0 out_seq=0 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9a7be00640 10 client.16632 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9a7be00640 1 -- 172.21.15.7:0/4092764840 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bf718949e0 con 0x55bf719a7810 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9a7be00640 10 client.16632 handle_command_reply: tid=0 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 client.16632 shutdown 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 msgr2=0x55bf71944070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000c00 tx=0x7f9a80012590 comp rx=0 tx=0).stop 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 shutdown_connections 2024-04-20T21:41:32.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bf71930880 0x55bf719445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55bf719a7810 0x55bf719a9c00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55bf71927ca0 0x55bf71944070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 --2- 172.21.15.7:0/4092764840 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55bf71927380 0x55bf71943b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 >> 172.21.15.7:0/4092764840 conn(0x55bf71921ab0 msgr2=0x55bf71920b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9a7be00640 0 client.16632 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 shutdown_connections 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 1 -- 172.21.15.7:0/4092764840 wait complete. 2024-04-20T21:41:32.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.956+0000 7f9aa017a740 20 client.16632 trim_cache size 0 max 16384 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout: "d1b19819-c8cf-4e31-b92c-4bd3c7f0ef21": { 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:41:32.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d1b19819-c8cf-4e31-b92c-4bd3c7f0ef21", 2024-04-20T21:41:32.965 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:41:32.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:41:32.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:41:32.965 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 msgr2=0x7f9a7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 0x7f9a7c0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x55bf71926e50 tx=0x7f9a88009700 comp rx=0 tx=0).stop 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 msgr2=0x7f9a981a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a981a48b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e9f0 tx=0x7f9a8800eeb0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 shutdown_connections 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a98112bf0 0x7f9a9819eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a98076db0 0x7f9a981a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f9a7c0a3d20 0x7f9a7c0a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 --2- 172.21.15.7:0/3568912347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a98076490 0x7f9a981a48b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:41:32.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 >> 172.21.15.7:0/3568912347 conn(0x7f9a980ffa00 msgr2=0x7f9a98114310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:41:32.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 shutdown_connections 2024-04-20T21:41:32.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:41:32.957+0000 7f9a9ca00640 1 -- 172.21.15.7:0/3568912347 wait complete. 2024-04-20T21:41:32.973 INFO:tasks.cephfs.filesystem:scrub status for tag:d1b19819-c8cf-4e31-b92c-4bd3c7f0ef21 - {'path': '/', 'tag': 'd1b19819-c8cf-4e31-b92c-4bd3c7f0ef21', 'options': 'recursive,force'} 2024-04-20T21:41:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:33 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:33.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:33 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:33 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:33.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:33 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:33 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:33 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:33.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:33 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:33.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:33 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:33.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:33 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:33 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:33 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:33 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:34 smithi016 ceph-mon[21409]: pgmap v254: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:41:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:41:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:34 smithi186 ceph-mon[22157]: pgmap v254: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:41:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:34 smithi007 ceph-mon[16867]: pgmap v254: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:41:36.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:36 smithi016 ceph-mon[21409]: pgmap v255: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-20T21:41:36.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:36 smithi186 ceph-mon[22157]: pgmap v255: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-20T21:41:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:36 smithi007 ceph-mon[16867]: pgmap v255: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 4.1 KiB/s wr, 6 op/s 2024-04-20T21:41:38.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:38 smithi016 ceph-mon[21409]: pgmap v256: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-20T21:41:38.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:38 smithi186 ceph-mon[22157]: pgmap v256: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-20T21:41:38.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:38 smithi007 ceph-mon[16867]: pgmap v256: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.1 KiB/s wr, 5 op/s 2024-04-20T21:41:40.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:40 smithi016 ceph-mon[21409]: pgmap v257: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-20T21:41:40.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:40 smithi186 ceph-mon[22157]: pgmap v257: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-20T21:41:40.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:40 smithi007 ceph-mon[16867]: pgmap v257: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-20T21:41:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:42 smithi016 ceph-mon[21409]: pgmap v258: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T21:41:42.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:42 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:42.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:42 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:42 smithi186 ceph-mon[22157]: pgmap v258: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T21:41:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:42 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:42.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:42 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:42 smithi007 ceph-mon[16867]: pgmap v258: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-20T21:41:42.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:42 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:42.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:42 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:43.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:43 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:41:43.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:43 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:41:43.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:43 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:41:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:44 smithi016 ceph-mon[21409]: pgmap v259: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:44 smithi186 ceph-mon[22157]: pgmap v259: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:44.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:44 smithi007 ceph-mon[16867]: pgmap v259: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:45.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:45 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:45.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:45 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:45 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:41:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:46 smithi016 ceph-mon[21409]: pgmap v260: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:46 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:46.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:46 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:46.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:46 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:41:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:46 smithi186 ceph-mon[22157]: pgmap v260: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:46 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:46 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:46.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:46 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:41:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:46 smithi007 ceph-mon[16867]: pgmap v260: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-20T21:41:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:46 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:41:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:46 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:41:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:46 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:41:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:48 smithi186 ceph-mon[22157]: pgmap v261: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:41:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:48 smithi007 ceph-mon[16867]: pgmap v261: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:41:48.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:48 smithi016 ceph-mon[21409]: pgmap v261: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-20T21:41:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:50 smithi186 ceph-mon[22157]: pgmap v262: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:41:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:50 smithi007 ceph-mon[16867]: pgmap v262: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:41:50.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:50 smithi016 ceph-mon[21409]: pgmap v262: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-20T21:41:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:52 smithi186 ceph-mon[22157]: pgmap v263: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:52 smithi007 ceph-mon[16867]: pgmap v263: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:52.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:52 smithi016 ceph-mon[21409]: pgmap v263: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:54 smithi186 ceph-mon[22157]: pgmap v264: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:41:54.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:54 smithi007 ceph-mon[16867]: pgmap v264: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:41:54.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:54 smithi016 ceph-mon[21409]: pgmap v264: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:41:56.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:56 smithi007 ceph-mon[16867]: pgmap v265: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:41:56.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:56 smithi016 ceph-mon[21409]: pgmap v265: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:41:56.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:56 smithi186 ceph-mon[22157]: pgmap v265: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:41:58.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:41:58 smithi007 ceph-mon[16867]: pgmap v266: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:58.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:41:58 smithi016 ceph-mon[21409]: pgmap v266: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:41:58.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:41:58 smithi186 ceph-mon[22157]: pgmap v266: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:00.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:00 smithi007 ceph-mon[16867]: pgmap v267: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:00.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:00 smithi016 ceph-mon[21409]: pgmap v267: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:00.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:00 smithi186 ceph-mon[22157]: pgmap v267: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:02 smithi007 ceph-mon[16867]: pgmap v268: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:02.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:02 smithi016 ceph-mon[21409]: pgmap v268: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:02.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:02 smithi186 ceph-mon[22157]: pgmap v268: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:02.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:02.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 -- 172.21.15.7:0/2342828575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c074cb0 msgr2=0x7f372c10e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 --2- 172.21.15.7:0/2342828575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c074cb0 0x7f372c10e060 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3718009900 tx=0x7f371802bcd0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 -- 172.21.15.7:0/2342828575 shutdown_connections 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 --2- 172.21.15.7:0/2342828575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c10e5a0 0x7f372c1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 --2- 172.21.15.7:0/2342828575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c074cb0 0x7f372c10e060 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 --2- 172.21.15.7:0/2342828575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c074390 0x7f372c074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 -- 172.21.15.7:0/2342828575 >> 172.21.15.7:0/2342828575 conn(0x7f372c0fd8b0 msgr2=0x7f372c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 -- 172.21.15.7:0/2342828575 shutdown_connections 2024-04-20T21:42:03.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.200+0000 7f3730a00640 1 -- 172.21.15.7:0/2342828575 wait complete. 2024-04-20T21:42:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.201+0000 7f3730a00640 1 Processor -- start 2024-04-20T21:42:03.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 -- start start 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c10e5a0 0x7f372c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c19a930 0x7f372c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f372c114360 con 0x7f372c19a930 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1141e0 con 0x7f372c10e5a0 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3730a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1144e0 con 0x7f372c074390 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f372a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f372a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50772/0 (socket says 172.21.15.7:50772) 2024-04-20T21:42:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f372a800640 1 -- 172.21.15.7:0/2037848218 learned_addr learned my addr 172.21.15.7:0/2037848218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f372b200640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c19a930 0x7f372c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.202+0000 7f3729e00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c10e5a0 0x7f372c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 -- 172.21.15.7:0/2037848218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c10e5a0 msgr2=0x7f372c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c10e5a0 0x7f372c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 -- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c19a930 msgr2=0x7f372c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c19a930 0x7f372c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 -- 172.21.15.7:0/2037848218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c19b450 con 0x7f372c074390 2024-04-20T21:42:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f372a800640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f3723400640 1 -- 172.21.15.7:0/2037848218 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c00cd80 con 0x7f372c074390 2024-04-20T21:42:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.203+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c19b740 con 0x7f372c074390 2024-04-20T21:42:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.204+0000 7f3723400640 1 -- 172.21.15.7:0/2037848218 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f371c010910 con 0x7f372c074390 2024-04-20T21:42:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.204+0000 7f3723400640 1 -- 172.21.15.7:0/2037848218 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0188f0 con 0x7f372c074390 2024-04-20T21:42:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.204+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1a79e0 con 0x7f372c074390 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 Processor -- start 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 -- start start 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb5c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb5c880 0x5571cbb5ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571cbab9150 con 0x5571cbb5c880 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571cba5f370 con 0x5571cbb53380 2024-04-20T21:42:03.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f37341a1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5571cbab5110 con 0x5571cbb53ca0 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.207+0000 7f3723400640 1 -- 172.21.15.7:0/2037848218 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f371c010ac0 con 0x7f372c074390 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56260/0 (socket says 172.21.15.7:56260) 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 -- 172.21.15.7:0/1435447152 learned_addr learned my addr 172.21.15.7:0/1435447152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f3729e00640 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb5c880 0x5571cbb5ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 -- 172.21.15.7:0/1435447152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 msgr2=0x5571cbb5c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb5c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 -- 172.21.15.7:0/1435447152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb5c880 msgr2=0x5571cbb5ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb5c880 0x5571cbb5ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 -- 172.21.15.7:0/1435447152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cb8957d0 con 0x5571cbb53380 2024-04-20T21:42:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.208+0000 7f372b200640 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3724009f70 tx=0x7f3724010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1150268f051302f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f3717400640 1 -- 172.21.15.7:0/1435447152 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724016070 con 0x5571cbb53380 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f3717400640 1 -- 172.21.15.7:0/1435447152 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3724004d20 con 0x5571cbb53380 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f3717400640 1 -- 172.21.15.7:0/1435447152 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724012950 con 0x5571cbb53380 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 -- 172.21.15.7:0/1435447152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 msgr2=0x5571cbb53760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3724009f70 tx=0x7f3724010800 comp rx=0 tx=0).stop 2024-04-20T21:42:03.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f3723400640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 0x7f37100a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 -- 172.21.15.7:0/1435447152 shutdown_connections 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb5c880 0x5571cbb5ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb5c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 --2- 172.21.15.7:0/1435447152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53380 0x5571cbb53760 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.209+0000 7f37341a1740 1 -- 172.21.15.7:0/1435447152 >> 172.21.15.7:0/1435447152 conn(0x5571cbb4dab0 msgr2=0x5571cbb4de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f37341a1740 1 -- 172.21.15.7:0/1435447152 shutdown_connections 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f3723400640 1 -- 172.21.15.7:0/2037848218 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f371c0d0af0 con 0x7f372c074390 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f37341a1740 1 -- 172.21.15.7:0/1435447152 wait complete. 2024-04-20T21:42:03.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f372b200640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 0x7f37100a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f37341a1740 1 -- start start 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.210+0000 7f37341a1740 1 Processor -- start 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f372b200640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 0x7f37100a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3724009f70 tx=0x7f3724010fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53ca0 0x5571cbb59e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb5c880 0x5571cbb5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5571cbb5f670 con 0x5571cbb53380 2024-04-20T21:42:03.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571cba72740 con 0x5571cbb53ca0 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5571cbb5f7f0 con 0x5571cbb5c880 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f37341a1740 10 client.0 _ll_register_callbacks cb 0x5571cba6db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f372a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f372a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45788/0 (socket says 172.21.15.7:45788) 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f372a800640 1 -- 172.21.15.7:0/2582195165 learned_addr learned my addr 172.21.15.7:0/2582195165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f372b200640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb5c880 0x5571cbb5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.211+0000 7f3729e00640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53ca0 0x5571cbb59e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 -- 172.21.15.7:0/2582195165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb5c880 msgr2=0x5571cbb5a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb5c880 0x5571cbb5a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 -- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53ca0 msgr2=0x5571cbb59e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53ca0 0x5571cbb59e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 -- 172.21.15.7:0/2582195165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cb7cb3f0 con 0x5571cbb53380 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f372a800640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f371c0ce860 tx=0x7f371c09cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d99d0118424f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f370be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c09bfb0 con 0x5571cbb53380 2024-04-20T21:42:03.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f371c09c150 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3724004d20 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.212+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f371c010430 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 10 client.16644 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571cb78b0f0 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3724004d20 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f371c09bc20 con 0x5571cbb53380 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 10 client.16644 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:03.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 20 client.16644 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbb6cba0 0x5571cbb6ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 4 client.16644 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.213+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571cbac6b70 con 0x5571cbb6cba0 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.214+0000 7f3729e00640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbb6cba0 0x5571cbb6ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.214+0000 7f3729e00640 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbb6cba0 0x5571cbb6ef90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884b205057429e78 server_cookie=bc7275dc8b68b5ba in_seq=0 out_seq=0 2024-04-20T21:42:03.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.214+0000 7f370be00640 10 client.16644 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:03.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f370be00640 1 -- 172.21.15.7:0/2582195165 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571cbac6b70 con 0x5571cbb6cba0 2024-04-20T21:42:03.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f370be00640 10 client.16644 handle_command_reply: tid=0 2024-04-20T21:42:03.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 client.16644 shutdown 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 msgr2=0x5571cbb598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f371c0ce860 tx=0x7f371c09cb60 comp rx=0 tx=0).stop 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 shutdown_connections 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb53ca0 0x5571cbb59e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbb6cba0 0x5571cbb6ef90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb5c880 0x5571cbb5a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 --2- 172.21.15.7:0/2582195165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb598f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 >> 172.21.15.7:0/2582195165 conn(0x5571cbb4dab0 msgr2=0x5571cbb4f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f370be00640 0 client.16644 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 shutdown_connections 2024-04-20T21:42:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.215+0000 7f37341a1740 1 -- 172.21.15.7:0/2582195165 wait complete. 2024-04-20T21:42:03.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.216+0000 7f37341a1740 20 client.16644 trim_cache size 0 max 16384 2024-04-20T21:42:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.235+0000 7f37341a1740 1 Processor -- start 2024-04-20T21:42:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.235+0000 7f37341a1740 1 -- start start 2024-04-20T21:42:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb5c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb68530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1b2c10 con 0x5571cbb53380 2024-04-20T21:42:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1b04d0 con 0x5571cbb5c880 2024-04-20T21:42:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f37341a1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1ae340 con 0x5571cbb53ca0 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f372a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb5c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f3729e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f372b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb68530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f3729e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50796/0 (socket says 172.21.15.7:50796) 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.236+0000 7f3729e00640 1 -- 172.21.15.7:0/2906546159 learned_addr learned my addr 172.21.15.7:0/2906546159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 -- 172.21.15.7:0/2906546159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 msgr2=0x5571cbb68530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb68530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 -- 172.21.15.7:0/2906546159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 msgr2=0x5571cbb5c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb5c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 -- 172.21.15.7:0/2906546159 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571cbb53ad0 con 0x5571cbb53ca0 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3729e00640 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3718036040 tx=0x7f371802e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760517c538a4b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.237+0000 7f3716a00640 1 -- 172.21.15.7:0/2906546159 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718048b50 con 0x5571cbb53ca0 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f3716a00640 1 -- 172.21.15.7:0/2906546159 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3718048cf0 con 0x5571cbb53ca0 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f3716a00640 1 -- 172.21.15.7:0/2906546159 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37180046c0 con 0x5571cbb53ca0 2024-04-20T21:42:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 -- 172.21.15.7:0/2906546159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 msgr2=0x5571cbb58850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3718036040 tx=0x7f371802e040 comp rx=0 tx=0).stop 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 -- 172.21.15.7:0/2906546159 shutdown_connections 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb68530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53ca0 0x5571cbb58850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 --2- 172.21.15.7:0/2906546159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53380 0x5571cbb5c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 -- 172.21.15.7:0/2906546159 >> 172.21.15.7:0/2906546159 conn(0x5571cbb4dab0 msgr2=0x5571cbb4cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 -- 172.21.15.7:0/2906546159 shutdown_connections 2024-04-20T21:42:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.238+0000 7f37341a1740 1 -- 172.21.15.7:0/2906546159 wait complete. 2024-04-20T21:42:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 -- start start 2024-04-20T21:42:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 Processor -- start 2024-04-20T21:42:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 0x5571cbb60580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1ae990 con 0x5571cbb53ca0 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.239+0000 7f37341a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1afe00 con 0x5571cbb5c880 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f37341a1740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f372c1af060 con 0x5571cbb53380 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f37341a1740 10 client.0 _ll_register_callbacks cb 0x7f371c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f372a800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 0x5571cbb60580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45800/0 (socket says 172.21.15.7:45800) 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 -- 172.21.15.7:0/1313940968 learned_addr learned my addr 172.21.15.7:0/1313940968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f372b200640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 -- 172.21.15.7:0/1313940968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 msgr2=0x5571cbb60580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 0x5571cbb60580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 -- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 msgr2=0x5571cbb61000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb61000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f3729e00640 1 -- 172.21.15.7:0/1313940968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571cbbda950 con 0x5571cbb53ca0 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.240+0000 7f372a800640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 0x5571cbb60580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f3729e00640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f371802c2e0 tx=0x7f3718005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cea3abb97c44231 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f370be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371802c9e0 con 0x5571cbb53ca0 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f37180046c0 con 0x5571cbb53ca0 2024-04-20T21:42:03.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3718048cf0 con 0x5571cbb53ca0 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3718060050 con 0x5571cbb53ca0 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f37341a1740 10 client.16650 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.241+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5571cbbdbc40 con 0x5571cbb53ca0 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3718048cf0 con 0x5571cbb53ca0 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3718048360 con 0x5571cbb53ca0 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 10 client.16650 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:03.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 20 client.16650 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbbdc3e0 0x5571cbbde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 4 client.16650 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571cbac09e0 con 0x5571cbbdc3e0 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.242+0000 7f372a800640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbbdc3e0 0x5571cbbde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f372a800640 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbbdc3e0 0x5571cbbde800 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f448c1b9d854302 server_cookie=261883d28e8b7cb9 in_seq=0 out_seq=0 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f370be00640 10 client.16650 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f370be00640 1 -- 172.21.15.7:0/1313940968 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5571cbac09e0 con 0x5571cbbdc3e0 2024-04-20T21:42:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f370be00640 10 client.16650 handle_command_reply: tid=0 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 client.16650 shutdown 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 msgr2=0x5571cbb60ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f371802c2e0 tx=0x7f3718005900 comp rx=0 tx=0).stop 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 shutdown_connections 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571cbb5c880 0x5571cbb61000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5571cbbdc3e0 0x5571cbbde800 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5571cbb53ca0 0x5571cbb60ac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 --2- 172.21.15.7:0/1313940968 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5571cbb53380 0x5571cbb60580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 >> 172.21.15.7:0/1313940968 conn(0x5571cbb4dab0 msgr2=0x5571cbb598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.243+0000 7f370be00640 0 client.16650 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 shutdown_connections 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f37341a1740 1 -- 172.21.15.7:0/1313940968 wait complete. 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f37341a1740 20 client.16650 trim_cache size 0 max 16384 2024-04-20T21:42:03.251 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:03.252 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:42:03.252 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:42:03.252 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 msgr2=0x7f37100a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 0x7f37100a6260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3724009f70 tx=0x7f3724010fd0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 msgr2=0x7f372c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f371c00e9f0 tx=0x7f371c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.244+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 shutdown_connections 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f37100a3da0 0x7f37100a6260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f372c10e5a0 0x7f372c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372c19a930 0x7f372c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 --2- 172.21.15.7:0/2037848218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f372c074390 0x7f372c1a0780 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 >> 172.21.15.7:0/2037848218 conn(0x7f372c0fd8b0 msgr2=0x7f372c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 shutdown_connections 2024-04-20T21:42:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.245+0000 7f3730a00640 1 -- 172.21.15.7:0/2037848218 wait complete. 2024-04-20T21:42:03.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:42:03.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 -- 172.21.15.7:0/3313364379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044100de0 msgr2=0x7f80441011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 --2- 172.21.15.7:0/3313364379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044100de0 0x7f80441011c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8034009be0 tx=0x7f803402bfd0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 -- 172.21.15.7:0/3313364379 shutdown_connections 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 --2- 172.21.15.7:0/3313364379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f8044105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 --2- 172.21.15.7:0/3313364379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80441061e0 0x7f8044106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 --2- 172.21.15.7:0/3313364379 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8044100de0 0x7f80441011c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.487+0000 7f8049600640 1 -- 172.21.15.7:0/3313364379 >> 172.21.15.7:0/3313364379 conn(0x7f80440fc7f0 msgr2=0x7f80440fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.488+0000 7f8049600640 1 -- 172.21.15.7:0/3313364379 shutdown_connections 2024-04-20T21:42:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.488+0000 7f8049600640 1 -- 172.21.15.7:0/3313364379 wait complete. 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 Processor -- start 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 -- start start 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80441061e0 0x7f80441a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8044118850 con 0x7f8044100de0 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80441186d0 con 0x7f8044101700 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.489+0000 7f8049600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f80441189d0 con 0x7f80441061e0 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f8043e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f8048c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80441061e0 0x7f80441a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f8043e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45810/0 (socket says 172.21.15.7:45810) 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56306/0 (socket says 172.21.15.7:56306) 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f8043e00640 1 -- 172.21.15.7:0/3982731619 learned_addr learned my addr 172.21.15.7:0/3982731619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 -- 172.21.15.7:0/3982731619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80441061e0 msgr2=0x7f80441a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80441061e0 0x7f80441a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 -- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 msgr2=0x7f804419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f803b400640 1 -- 172.21.15.7:0/3982731619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80441a3e00 con 0x7f8044101700 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.490+0000 7f8043e00640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f803b400640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e7c0 tx=0x7f802c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c009530 con 0x7f8044101700 2024-04-20T21:42:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c00ccc0 con 0x7f8044101700 2024-04-20T21:42:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004a00 con 0x7f8044101700 2024-04-20T21:42:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80441a40f0 con 0x7f8044101700 2024-04-20T21:42:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.491+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80441ab980 con 0x7f8044101700 2024-04-20T21:42:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.492+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804404fd60 con 0x7f8044101700 2024-04-20T21:42:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.494+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f802c004ba0 con 0x7f8044101700 2024-04-20T21:42:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.496+0000 7f8041600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 0x7f80280a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.496+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f802c0d06b0 con 0x7f8044101700 2024-04-20T21:42:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.496+0000 7f8043e00640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 0x7f80280a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.497+0000 7f8043e00640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 0x7f80280a6250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8034009330 tx=0x7f8034009280 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.500+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f802c09cd10 con 0x7f8044101700 2024-04-20T21:42:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.750+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80441abb50 con 0x7f8044101700 2024-04-20T21:42:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.752+0000 7f8041600640 1 -- 172.21.15.7:0/3982731619 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f802c09c610 con 0x7f8044101700 2024-04-20T21:42:03.751 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:42:03.751 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:42:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 msgr2=0x7f80280a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 0x7f80280a6250 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8034009330 tx=0x7f8034009280 comp rx=0 tx=0).stop 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 msgr2=0x7f804419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e7c0 tx=0x7f802c00ec80 comp rx=0 tx=0).stop 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 shutdown_connections 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80280a3d90 0x7f80280a6250 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8044101700 0x7f804419f350 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80441061e0 0x7f80441a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 --2- 172.21.15.7:0/3982731619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8044100de0 0x7f804419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 >> 172.21.15.7:0/3982731619 conn(0x7f80440fc7f0 msgr2=0x7f8044103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 shutdown_connections 2024-04-20T21:42:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.754+0000 7f8049600640 1 -- 172.21.15.7:0/3982731619 wait complete. 2024-04-20T21:42:03.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:42:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 -- 172.21.15.7:0/1321889522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341041f0 msgr2=0x7fac341045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 --2- 172.21.15.7:0/1321889522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341041f0 0x7fac341045d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ca0 tx=0x7fac2402c100 comp rx=0 tx=0).stop 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 -- 172.21.15.7:0/1321889522 shutdown_connections 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 --2- 172.21.15.7:0/1321889522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac34104ba0 0x7fac34108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 --2- 172.21.15.7:0/1321889522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac34110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 --2- 172.21.15.7:0/1321889522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341041f0 0x7fac341045d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 -- 172.21.15.7:0/1321889522 >> 172.21.15.7:0/1321889522 conn(0x7fac340ffa50 msgr2=0x7fac34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 -- 172.21.15.7:0/1321889522 shutdown_connections 2024-04-20T21:42:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.990+0000 7fac39600640 1 -- 172.21.15.7:0/1321889522 wait complete. 2024-04-20T21:42:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.991+0000 7fac39600640 1 Processor -- start 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 -- start start 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 0x7fac3419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac34114370 con 0x7fac341092c0 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac341141f0 con 0x7fac3419ae50 2024-04-20T21:42:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac39600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fac341144f0 con 0x7fac341a0ca0 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac2b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac2b400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50834/0 (socket says 172.21.15.7:50834) 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45824/0 (socket says 172.21.15.7:45824) 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac2b400640 1 -- 172.21.15.7:0/3481060780 learned_addr learned my addr 172.21.15.7:0/3481060780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.992+0000 7fac38c00640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 0x7fac3419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 -- 172.21.15.7:0/3481060780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 msgr2=0x7fac3419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 0x7fac3419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 -- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 msgr2=0x7fac341a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 -- 172.21.15.7:0/3481060780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac341a7620 con 0x7fac341a0ca0 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac33e00640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac38c00640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 0x7fac3419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac2b400640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9f0 tx=0x7fac1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac31600640 1 -- 172.21.15.7:0/3481060780 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00ccc0 con 0x7fac341a0ca0 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac31600640 1 -- 172.21.15.7:0/3481060780 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1c005500 con 0x7fac341a0ca0 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.993+0000 7fac31600640 1 -- 172.21.15.7:0/3481060780 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c010690 con 0x7fac341a0ca0 2024-04-20T21:42:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.994+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341a7850 con 0x7fac341a0ca0 2024-04-20T21:42:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.994+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341a7d20 con 0x7fac341a0ca0 2024-04-20T21:42:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 Processor -- start 2024-04-20T21:42:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 -- start start 2024-04-20T21:42:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf129c4050 con 0x55cf12a5f170 2024-04-20T21:42:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf1296a490 con 0x55cf12a5fa90 2024-04-20T21:42:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.998+0000 7fac3d970740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cf129c0010 con 0x55cf12a68670 2024-04-20T21:42:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac31600640 1 -- 172.21.15.7:0/3481060780 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fac1c0046c0 con 0x7fac341a0ca0 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac38c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac38c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45832/0 (socket says 172.21.15.7:45832) 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac38c00640 1 -- 172.21.15.7:0/3518520843 learned_addr learned my addr 172.21.15.7:0/3518520843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 -- 172.21.15.7:0/3518520843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 msgr2=0x55cf12a6aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac2b400640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 -- 172.21.15.7:0/3518520843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 msgr2=0x55cf12a5f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac33e00640 1 -- 172.21.15.7:0/3518520843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf12972070 con 0x55cf12a5fa90 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac2b400640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:03.999+0000 7fac38c00640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac33e00640 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a68130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fac24008000 tx=0x7fac24009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdcb5d6ced9a9861 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac29200640 1 -- 172.21.15.7:0/3518520843 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24043070 con 0x55cf12a5fa90 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac29200640 1 -- 172.21.15.7:0/3518520843 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac24003970 con 0x55cf12a5fa90 2024-04-20T21:42:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac29200640 1 -- 172.21.15.7:0/3518520843 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24039040 con 0x55cf12a5fa90 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac31600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 0x7fac180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.000+0000 7fac31600640 1 -- 172.21.15.7:0/3481060780 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fac1c0d0b90 con 0x7fac341a0ca0 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac38c00640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 0x7fac180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 -- 172.21.15.7:0/3518520843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 msgr2=0x55cf12a68130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a68130 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fac24008000 tx=0x7fac24009940 comp rx=0 tx=0).stop 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 -- 172.21.15.7:0/3518520843 shutdown_connections 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a68130 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 --2- 172.21.15.7:0/3518520843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a5f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 -- 172.21.15.7:0/3518520843 >> 172.21.15.7:0/3518520843 conn(0x55cf12a598a0 msgr2=0x55cf12a59c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 -- 172.21.15.7:0/3518520843 shutdown_connections 2024-04-20T21:42:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac3d970740 1 -- 172.21.15.7:0/3518520843 wait complete. 2024-04-20T21:42:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.001+0000 7fac38c00640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 0x7fac180a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005f60 tx=0x7fac2c005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.002+0000 7fac3d970740 1 -- start start 2024-04-20T21:42:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.002+0000 7fac3d970740 1 Processor -- start 2024-04-20T21:42:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.002+0000 7fac3d970740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a65660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55cf12a6b400 con 0x55cf12a68670 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf1297d860 con 0x55cf12a5fa90 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55cf12a6b580 con 0x55cf12a5f170 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac3d970740 10 client.0 _ll_register_callbacks cb 0x55cf1294bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac38c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac38c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45838/0 (socket says 172.21.15.7:45838) 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac38c00640 1 -- 172.21.15.7:0/667305266 learned_addr learned my addr 172.21.15.7:0/667305266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac33e00640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a65660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 -- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 msgr2=0x55cf12a65660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a65660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 -- 172.21.15.7:0/667305266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 msgr2=0x55cf12a660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.003+0000 7fac2b400640 1 -- 172.21.15.7:0/667305266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf12981950 con 0x55cf12a5fa90 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac2b400640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a65ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0ce2a0 tx=0x7fac1c09c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8604e44d3e75d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac21600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0d5040 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fac1c0a52d0 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac24003970 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac1c0ce720 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac3d970740 10 client.26879 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf1294f820 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.004+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac24003970 con 0x55cf12a5fa90 2024-04-20T21:42:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fac1c09c060 con 0x55cf12a5fa90 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 10 client.26879 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 10 client.26879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 20 client.26879 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12a784d0 0x55cf12a7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 4 client.26879 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf129d1a70 con 0x55cf12a784d0 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.005+0000 7fac33e00640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12a784d0 0x55cf12a7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.006+0000 7fac33e00640 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12a784d0 0x55cf12a7a900 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aa7e0a53c7d1457 server_cookie=2918ec9ba248a4e7 in_seq=0 out_seq=0 2024-04-20T21:42:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.006+0000 7fac21600640 10 client.26879 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac21600640 1 -- 172.21.15.7:0/667305266 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf129d1a70 con 0x55cf12a784d0 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac21600640 10 client.26879 handle_command_reply: tid=0 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 client.26879 shutdown 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 msgr2=0x55cf12a65ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a65ba0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0ce2a0 tx=0x7fac1c09c6b0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 shutdown_connections 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12a784d0 0x55cf12a7a900 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a65ba0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 --2- 172.21.15.7:0/667305266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a65660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac21600640 0 client.26879 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 >> 172.21.15.7:0/667305266 conn(0x55cf12a598a0 msgr2=0x55cf12a5b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 shutdown_connections 2024-04-20T21:42:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 1 -- 172.21.15.7:0/667305266 wait complete. 2024-04-20T21:42:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.007+0000 7fac3d970740 20 client.26879 trim_cache size 0 max 16384 2024-04-20T21:42:04.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.022+0000 7fac3d970740 1 Processor -- start 2024-04-20T21:42:04.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 -- start start 2024-04-20T21:42:04.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac341b2af0 con 0x55cf12a68670 2024-04-20T21:42:04.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac341b03b0 con 0x55cf12a5fa90 2024-04-20T21:42:04.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac3d970740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ae220 con 0x55cf12a5f170 2024-04-20T21:42:04.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac33e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac38c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac33e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50862/0 (socket says 172.21.15.7:50862) 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.023+0000 7fac33e00640 1 -- 172.21.15.7:0/3641065331 learned_addr learned my addr 172.21.15.7:0/3641065331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac2b400640 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 -- 172.21.15.7:0/3641065331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 msgr2=0x55cf12a742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 -- 172.21.15.7:0/3641065331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 msgr2=0x55cf12a65d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a65d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 -- 172.21.15.7:0/3641065331 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf126d63f0 con 0x55cf12a5f170 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac2b400640 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac33e00640 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac24009870 tx=0x7fac24005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5bd95268c22983d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac23e00640 1 -- 172.21.15.7:0/3641065331 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24056070 con 0x55cf12a5f170 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac23e00640 1 -- 172.21.15.7:0/3641065331 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac24032e50 con 0x55cf12a5f170 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.024+0000 7fac23e00640 1 -- 172.21.15.7:0/3641065331 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24039040 con 0x55cf12a5f170 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- 172.21.15.7:0/3641065331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 msgr2=0x55cf12a67d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac24009870 tx=0x7fac24005c50 comp rx=0 tx=0).stop 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- 172.21.15.7:0/3641065331 shutdown_connections 2024-04-20T21:42:04.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a68670 0x55cf12a65d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 --2- 172.21.15.7:0/3641065331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a5f170 0x55cf12a67d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- 172.21.15.7:0/3641065331 >> 172.21.15.7:0/3641065331 conn(0x55cf12a598a0 msgr2=0x55cf12a65620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- 172.21.15.7:0/3641065331 shutdown_connections 2024-04-20T21:42:04.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- 172.21.15.7:0/3641065331 wait complete. 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.025+0000 7fac3d970740 1 -- start start 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 Processor -- start 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac341ae870 con 0x55cf12a5f170 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac341afce0 con 0x55cf12a5fa90 2024-04-20T21:42:04.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fac341aef40 con 0x55cf12a68670 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac3d970740 10 client.0 _ll_register_callbacks cb 0x7fac1c098e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45848/0 (socket says 172.21.15.7:45848) 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac33e00640 1 -- 172.21.15.7:0/2639988560 learned_addr learned my addr 172.21.15.7:0/2639988560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac2b400640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.026+0000 7fac38c00640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 -- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 msgr2=0x55cf12a6c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 -- 172.21.15.7:0/2639988560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 msgr2=0x55cf12a6bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a6bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 -- 172.21.15.7:0/2639988560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf127a07d0 con 0x55cf12a5f170 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac33e00640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac24004af0 tx=0x7fac24009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbd923a3aa3b8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac21600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.027+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24056070 con 0x55cf12a5f170 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fac24050ae0 con 0x55cf12a5f170 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac24032e50 con 0x55cf12a5f170 2024-04-20T21:42:04.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac24039070 con 0x55cf12a5f170 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 10 client.16674 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf12ade8e0 con 0x55cf12a5f170 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac24032e50 con 0x55cf12a5f170 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fac240347f0 con 0x55cf12a5f170 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 10 client.16674 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 10 client.16674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:42:04.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 20 client.16674 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.028+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12adebc0 0x55cf12ae0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.029+0000 7fac3d970740 4 client.16674 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.029+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf129cb8e0 con 0x55cf12adebc0 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.029+0000 7fac2b400640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12adebc0 0x55cf12ae0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.029+0000 7fac2b400640 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12adebc0 0x55cf12ae0fb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b04d546e88ab18b server_cookie=c806e1395740a324 in_seq=0 out_seq=0 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.029+0000 7fac21600640 10 client.16674 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac21600640 1 -- 172.21.15.7:0/2639988560 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf129cb8e0 con 0x55cf12adebc0 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac21600640 10 client.16674 handle_command_reply: tid=0 2024-04-20T21:42:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 client.16674 shutdown 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 msgr2=0x55cf12a70c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fac24004af0 tx=0x7fac24009800 comp rx=0 tx=0).stop 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 shutdown_connections 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x55cf12adebc0 0x55cf12ae0fb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf12a5fa90 0x55cf12a6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55cf12a68670 0x55cf12a6c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 --2- 172.21.15.7:0/2639988560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55cf12a5f170 0x55cf12a70c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac21600640 0 client.16674 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:04.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 >> 172.21.15.7:0/2639988560 conn(0x55cf12a598a0 msgr2=0x55cf12a58960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 shutdown_connections 2024-04-20T21:42:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 1 -- 172.21.15.7:0/2639988560 wait complete. 2024-04-20T21:42:04.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.030+0000 7fac3d970740 20 client.16674 trim_cache size 0 max 16384 2024-04-20T21:42:04.037 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 msgr2=0x7fac180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 0x7fac180a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005f60 tx=0x7fac2c005f90 comp rx=0 tx=0).stop 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 msgr2=0x7fac3419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e9f0 tx=0x7fac1c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 shutdown_connections 2024-04-20T21:42:04.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fac180a3da0 0x7fac180a6260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac3419ae50 0x7fac3419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fac341a0ca0 0x7fac3419a910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 --2- 172.21.15.7:0/3481060780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac341092c0 0x7fac341a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 >> 172.21.15.7:0/3481060780 conn(0x7fac340ffa50 msgr2=0x7fac34100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 shutdown_connections 2024-04-20T21:42:04.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.031+0000 7fac39600640 1 -- 172.21.15.7:0/3481060780 wait complete. 2024-04-20T21:42:04.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:42:04.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.269+0000 7fdbeee00640 1 -- 172.21.15.7:0/2027093280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 msgr2=0x7fdbe8110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.269+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2027093280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe8110e50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009ca0 tx=0x7fdbe002c0f0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 -- 172.21.15.7:0/2027093280 shutdown_connections 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2027093280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe8104b60 0x7fdbe8108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2027093280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe8110e50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2027093280 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe81041b0 0x7fdbe8104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 -- 172.21.15.7:0/2027093280 >> 172.21.15.7:0/2027093280 conn(0x7fdbe80ffa30 msgr2=0x7fdbe8101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 -- 172.21.15.7:0/2027093280 shutdown_connections 2024-04-20T21:42:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.270+0000 7fdbeee00640 1 -- 172.21.15.7:0/2027093280 wait complete. 2024-04-20T21:42:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.271+0000 7fdbeee00640 1 Processor -- start 2024-04-20T21:42:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.271+0000 7fdbeee00640 1 -- start start 2024-04-20T21:42:04.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 0x7fdbe81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe819a6c0 0x7fdbe819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8114320 con 0x7fdbe8109280 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81141a0 con 0x7fdbe81041b0 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeee00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81144a0 con 0x7fdbe819a6c0 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45870/0 (socket says 172.21.15.7:45870) 2024-04-20T21:42:04.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbed000640 1 -- 172.21.15.7:0/2953460456 learned_addr learned my addr 172.21.15.7:0/2953460456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 0x7fdbe81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.272+0000 7fdbee400640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe819a6c0 0x7fdbe819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 -- 172.21.15.7:0/2953460456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe819a6c0 msgr2=0x7fdbe819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe819a6c0 0x7fdbe819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 -- 172.21.15.7:0/2953460456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 msgr2=0x7fdbe81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 0x7fdbe81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 -- 172.21.15.7:0/2953460456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe819b2c0 con 0x7fdbe8109280 2024-04-20T21:42:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 0x7fdbe81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbed000640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400e9a0 tx=0x7fdbd400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbe6000640 1 -- 172.21.15.7:0/2953460456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400cd80 con 0x7fdbe8109280 2024-04-20T21:42:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbe6000640 1 -- 172.21.15.7:0/2953460456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40035f0 con 0x7fdbe8109280 2024-04-20T21:42:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbe6000640 1 -- 172.21.15.7:0/2953460456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4010670 con 0x7fdbe8109280 2024-04-20T21:42:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.273+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe81a73a0 con 0x7fdbe8109280 2024-04-20T21:42:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.274+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81a7800 con 0x7fdbe8109280 2024-04-20T21:42:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.277+0000 7fdbf31b6740 1 Processor -- start 2024-04-20T21:42:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.277+0000 7fdbf31b6740 1 -- start start 2024-04-20T21:42:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.277+0000 7fdbe6000640 1 -- 172.21.15.7:0/2953460456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdbd40037a0 con 0x7fdbe8109280 2024-04-20T21:42:04.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.277+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55883053b170 0x55883053b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.277+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x558830544130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5588304a0050 con 0x55883053ba90 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558830446490 con 0x55883053b170 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55883049c010 con 0x558830544670 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x558830544130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50892/0 (socket says 172.21.15.7:50892) 2024-04-20T21:42:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 -- 172.21.15.7:0/2493340219 learned_addr learned my addr 172.21.15.7:0/2493340219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x558830544130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45880/0 (socket says 172.21.15.7:45880) 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbee400640 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55883053b170 0x55883053b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 -- 172.21.15.7:0/2493340219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55883053b170 msgr2=0x55883053b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55883053b170 0x55883053b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 -- 172.21.15.7:0/2493340219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 msgr2=0x558830544130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x558830544130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.278+0000 7fdbed000640 1 -- 172.21.15.7:0/2493340219 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883044e070 con 0x558830544670 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbed000640 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40ce070 tx=0x7fdbd40d6ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2bc6430b1bf6ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbdaa00640 1 -- 172.21.15.7:0/2493340219 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40df070 con 0x558830544670 2024-04-20T21:42:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbe6000640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 0x7fdbd00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbdaa00640 1 -- 172.21.15.7:0/2493340219 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40d8de0 con 0x558830544670 2024-04-20T21:42:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbdaa00640 1 -- 172.21.15.7:0/2493340219 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40da7f0 con 0x558830544670 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbe6000640 1 -- 172.21.15.7:0/2953460456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fdbd40d09f0 con 0x7fdbe8109280 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.279+0000 7fdbee400640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 0x7fdbd00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2493340219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 msgr2=0x558830546a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdbd40ce070 tx=0x7fdbd40d6ef0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2493340219 shutdown_connections 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558830544670 0x558830546a60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x558830544130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2493340219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55883053b170 0x55883053b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2493340219 >> 172.21.15.7:0/2493340219 conn(0x5588305358a0 msgr2=0x558830535c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2493340219 shutdown_connections 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbee400640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 0x7fdbd00a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0004af0 tx=0x7fdbe0005c50 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.280+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2493340219 wait complete. 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 -- start start 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 Processor -- start 2024-04-20T21:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830541b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830542090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.281+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558830547460 con 0x55883053b170 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558830459860 con 0x558830544670 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5588305475e0 con 0x55883053ba90 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbf31b6740 10 client.0 _ll_register_callbacks cb 0x558830427fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45884/0 (socket says 172.21.15.7:45884) 2024-04-20T21:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 -- 172.21.15.7:0/2068620507 learned_addr learned my addr 172.21.15.7:0/2068620507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbee400640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830542090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbed000640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830541b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 -- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 msgr2=0x558830541b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830541b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 -- 172.21.15.7:0/2068620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 msgr2=0x558830542090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830542090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 -- 172.21.15.7:0/2068620507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883045d950 con 0x55883053b170 2024-04-20T21:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbeda00640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc012d20 tx=0x7fdbdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538e6c3f88912529 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.282+0000 7fdbcf400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc012110 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fdbdc004050 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd40d8de0 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0122b0 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbf31b6740 10 client.16686 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55883042b820 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.283+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd40d8de0 con 0x55883053b170 2024-04-20T21:42:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdbdc0104c0 con 0x55883053b170 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 10 client.16686 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 10 client.16686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 20 client.16686 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305546d0 0x558830556b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 4 client.16686 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588304ada70 con 0x5588305546d0 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.284+0000 7fdbed000640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305546d0 0x558830556b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.285+0000 7fdbed000640 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305546d0 0x558830556b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16c8754da9f92b92 server_cookie=f70e3c63b307ab9 in_seq=0 out_seq=0 2024-04-20T21:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.285+0000 7fdbcf400640 10 client.16686 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbcf400640 1 -- 172.21.15.7:0/2068620507 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588304ada70 con 0x5588305546d0 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbcf400640 10 client.16686 handle_command_reply: tid=0 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 client.16686 shutdown 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 msgr2=0x558830541610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc012d20 tx=0x7fdbdc00f790 comp rx=0 tx=0).stop 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 shutdown_connections 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305546d0 0x558830556b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830542090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830541b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/2068620507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830541610 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbcf400640 0 client.16686 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 >> 172.21.15.7:0/2068620507 conn(0x5588305358a0 msgr2=0x5588305375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 shutdown_connections 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 1 -- 172.21.15.7:0/2068620507 wait complete. 2024-04-20T21:42:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.286+0000 7fdbf31b6740 20 client.16686 trim_cache size 0 max 16384 2024-04-20T21:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.301+0000 7fdbf31b6740 1 Processor -- start 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.301+0000 7fdbf31b6740 1 -- start start 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830550380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830537400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81b2610 con 0x55883053b170 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81afed0 con 0x558830544670 2024-04-20T21:42:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81add40 con 0x55883053ba90 2024-04-20T21:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbed000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830550380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830537400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45894/0 (socket says 172.21.15.7:45894) 2024-04-20T21:42:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbeda00640 1 -- 172.21.15.7:0/3010236784 learned_addr learned my addr 172.21.15.7:0/3010236784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 -- 172.21.15.7:0/3010236784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 msgr2=0x558830550380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830550380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 -- 172.21.15.7:0/3010236784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 msgr2=0x558830543d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.302+0000 7fdbee400640 1 -- 172.21.15.7:0/3010236784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588301b23f0 con 0x558830544670 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbed000640 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830550380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbeda00640 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbee400640 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830537400 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002c700 tx=0x7fdbe0041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892b7be6e6fb5e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbda000640 1 -- 172.21.15.7:0/3010236784 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0039040 con 0x558830544670 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbda000640 1 -- 172.21.15.7:0/3010236784 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0003e50 con 0x558830544670 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.303+0000 7fdbda000640 1 -- 172.21.15.7:0/3010236784 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe004e460 con 0x558830544670 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- 172.21.15.7:0/3010236784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 msgr2=0x558830537400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830537400 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002c700 tx=0x7fdbe0041000 comp rx=0 tx=0).stop 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- 172.21.15.7:0/3010236784 shutdown_connections 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x558830537400 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053ba90 0x558830550380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/3010236784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053b170 0x558830543d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- 172.21.15.7:0/3010236784 >> 172.21.15.7:0/3010236784 conn(0x5588305358a0 msgr2=0x558830536cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- 172.21.15.7:0/3010236784 shutdown_connections 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- 172.21.15.7:0/3010236784 wait complete. 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.304+0000 7fdbf31b6740 1 -- start start 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 Processor -- start 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x55883053e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x55883053ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81ae390 con 0x55883053ba90 2024-04-20T21:42:04.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81af800 con 0x558830544670 2024-04-20T21:42:04.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81aea60 con 0x55883053b170 2024-04-20T21:42:04.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbf31b6740 10 client.0 _ll_register_callbacks cb 0x7fdbdc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x55883053e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbeda00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50920/0 (socket says 172.21.15.7:50920) 2024-04-20T21:42:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.305+0000 7fdbeda00640 1 -- 172.21.15.7:0/1077193744 learned_addr learned my addr 172.21.15.7:0/1077193744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbee400640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x55883053ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 -- 172.21.15.7:0/1077193744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 msgr2=0x55883053ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x55883053ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 -- 172.21.15.7:0/1077193744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 msgr2=0x55883053e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x55883053e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 -- 172.21.15.7:0/1077193744 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883027c7d0 con 0x55883053b170 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbeda00640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0106f0 tx=0x7fdbdc010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adba020966d232a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbcf400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc029040 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe0003e50 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.306+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fdbdc005040 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbdc005280 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 10 client.26854 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5588305bae70 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe0003e50 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdbdc02c070 con 0x55883053b170 2024-04-20T21:42:04.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 10 client.26854 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 10 client.26854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 20 client.26854 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305bb650 0x5588305bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 4 client.26854 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:04.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588304a78e0 con 0x5588305bb650 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.307+0000 7fdbed000640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305bb650 0x5588305bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.308+0000 7fdbed000640 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305bb650 0x5588305bda70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff52bdfda527bd75 server_cookie=fe4ac44f44154475 in_seq=0 out_seq=0 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.308+0000 7fdbcf400640 10 client.26854 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.308+0000 7fdbcf400640 1 -- 172.21.15.7:0/1077193744 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588304a78e0 con 0x5588305bb650 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.308+0000 7fdbcf400640 10 client.26854 handle_command_reply: tid=0 2024-04-20T21:42:04.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.308+0000 7fdbf31b6740 1 client.26854 shutdown 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 msgr2=0x55883053e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0106f0 tx=0x7fdbdc010720 comp rx=0 tx=0).stop 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 shutdown_connections 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x5588305bb650 0x5588305bda70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558830544670 0x55883053ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55883053ba90 0x55883053e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbcf400640 0 client.26854 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 --2- 172.21.15.7:0/1077193744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55883053b170 0x55883053e380 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 >> 172.21.15.7:0/1077193744 conn(0x5588305358a0 msgr2=0x558830541e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 shutdown_connections 2024-04-20T21:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 1 -- 172.21.15.7:0/1077193744 wait complete. 2024-04-20T21:42:04.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbf31b6740 20 client.26854 trim_cache size 0 max 16384 2024-04-20T21:42:04.318 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 msgr2=0x7fdbd00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.309+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 0x7fdbd00a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0004af0 tx=0x7fdbe0005c50 comp rx=0 tx=0).stop 2024-04-20T21:42:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 msgr2=0x7fdbe81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400e9a0 tx=0x7fdbd400ee60 comp rx=0 tx=0).stop 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 shutdown_connections 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbe819a6c0 0x7fdbe819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdbe8109280 0x7fdbe81a0b40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdbd00a3da0 0x7fdbd00a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 --2- 172.21.15.7:0/2953460456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbe81041b0 0x7fdbe81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 >> 172.21.15.7:0/2953460456 conn(0x7fdbe80ffa30 msgr2=0x7fdbe8101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 shutdown_connections 2024-04-20T21:42:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.310+0000 7fdbeee00640 1 -- 172.21.15.7:0/2953460456 wait complete. 2024-04-20T21:42:04.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:42:04.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 -- 172.21.15.7:0/1231687459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 msgr2=0x7f2cf41126b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 --2- 172.21.15.7:0/1231687459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf41126b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009890 tx=0x7f2ce002bb70 comp rx=0 tx=0).stop 2024-04-20T21:42:04.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 -- 172.21.15.7:0/1231687459 shutdown_connections 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 --2- 172.21.15.7:0/1231687459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 0x7f2cf4114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 --2- 172.21.15.7:0/1231687459 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf41126b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 --2- 172.21.15.7:0/1231687459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf4076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 -- 172.21.15.7:0/1231687459 >> 172.21.15.7:0/1231687459 conn(0x7f2cf40ffa00 msgr2=0x7f2cf4101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 -- 172.21.15.7:0/1231687459 shutdown_connections 2024-04-20T21:42:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.549+0000 7f2cf3200640 1 -- 172.21.15.7:0/1231687459 wait complete. 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.550+0000 7f2cf3200640 1 Processor -- start 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 -- start start 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 0x7f2cf41a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf4118740 con 0x7f2cf4112bf0 2024-04-20T21:42:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41185c0 con 0x7f2cf4076490 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.551+0000 7f2cf3200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41188c0 con 0x7f2cf4076db0 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 0x7f2cf41a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56436/0 (socket says 172.21.15.7:56436) 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1e00640 1 -- 172.21.15.7:0/3747839059 learned_addr learned my addr 172.21.15.7:0/3747839059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 -- 172.21.15.7:0/3747839059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 msgr2=0x7f2cf419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 -- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 msgr2=0x7f2cf41a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 0x7f2cf41a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 -- 172.21.15.7:0/3747839059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf41a3d70 con 0x7f2cf4076db0 2024-04-20T21:42:04.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.552+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf419f2c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00057c0 tx=0x7f2ce0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.553+0000 7f2ceaa00640 1 -- 172.21.15.7:0/3747839059 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0041020 con 0x7f2cf4076db0 2024-04-20T21:42:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.553+0000 7f2ceaa00640 1 -- 172.21.15.7:0/3747839059 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce002c9f0 con 0x7f2cf4076db0 2024-04-20T21:42:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.553+0000 7f2ceaa00640 1 -- 172.21.15.7:0/3747839059 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0039040 con 0x7f2cf4076db0 2024-04-20T21:42:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.553+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf41a4000 con 0x7f2cf4076db0 2024-04-20T21:42:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.553+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf41ab8f0 con 0x7f2cf4076db0 2024-04-20T21:42:04.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.556+0000 7f2cfbc34740 1 Processor -- start 2024-04-20T21:42:04.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.556+0000 7f2cfbc34740 1 -- start start 2024-04-20T21:42:04.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.556+0000 7f2ceaa00640 1 -- 172.21.15.7:0/3747839059 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f2ce0044020 con 0x7f2cf4076db0 2024-04-20T21:42:04.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c0c4050 con 0x55ed5c15f170 2024-04-20T21:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c06a490 con 0x55ed5c168670 2024-04-20T21:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c0c0010 con 0x55ed5c15fa90 2024-04-20T21:42:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45920/0 (socket says 172.21.15.7:45920) 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50934/0 (socket says 172.21.15.7:50934) 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf2800640 1 -- 172.21.15.7:0/1260529238 learned_addr learned my addr 172.21.15.7:0/1260529238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.557+0000 7f2cf1400640 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 -- 172.21.15.7:0/1260529238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 msgr2=0x55ed5c16aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 -- 172.21.15.7:0/1260529238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 msgr2=0x55ed5c15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 -- 172.21.15.7:0/1260529238 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5c072070 con 0x55ed5c15fa90 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf2800640 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009c80 tx=0x7f2ce4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3be227b409defc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2ceaa00640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 0x7f2cd80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cdf400640 1 -- 172.21.15.7:0/1260529238 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004230 con 0x55ed5c15fa90 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cdf400640 1 -- 172.21.15.7:0/1260529238 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce40043d0 con 0x55ed5c15fa90 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cdf400640 1 -- 172.21.15.7:0/1260529238 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4004d40 con 0x55ed5c15fa90 2024-04-20T21:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2ceaa00640 1 -- 172.21.15.7:0/3747839059 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f2ce00f1e10 con 0x7f2cf4076db0 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.558+0000 7f2cf2800640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 0x7f2cd80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1260529238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 msgr2=0x55ed5c168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009c80 tx=0x7f2ce4011810 comp rx=0 tx=0).stop 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1260529238 shutdown_connections 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c168130 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1260529238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1260529238 >> 172.21.15.7:0/1260529238 conn(0x55ed5c1598a0 msgr2=0x55ed5c159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cf2800640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 0x7f2cd80a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e780 tx=0x7f2cec010040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1260529238 shutdown_connections 2024-04-20T21:42:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.559+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1260529238 wait complete. 2024-04-20T21:42:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 -- start start 2024-04-20T21:42:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 Processor -- start 2024-04-20T21:42:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c1660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.560+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c16b460 con 0x55ed5c15f170 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c07d860 con 0x55ed5c168670 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ed5c16b5e0 con 0x55ed5c15fa90 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cfbc34740 10 client.0 _ll_register_callbacks cb 0x55ed5c04bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45932/0 (socket says 172.21.15.7:45932) 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 -- 172.21.15.7:0/3457284042 learned_addr learned my addr 172.21.15.7:0/3457284042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf2800640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c1660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 -- 172.21.15.7:0/3457284042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 msgr2=0x55ed5c165bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c165bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 -- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 msgr2=0x55ed5c1660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c1660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.561+0000 7f2cf1e00640 1 -- 172.21.15.7:0/3457284042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5c081950 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009fa0 tx=0x7f2ce4019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e43be48a9f51f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4028020 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f2ce4024620 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ce40043d0 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ce4033050 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cfbc34740 10 client.16704 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed5c04f820 con 0x55ed5c15f170 2024-04-20T21:42:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.562+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce40043d0 con 0x55ed5c15f170 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2ce4018070 con 0x55ed5c15f170 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 10 client.16704 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 20 client.16704 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c178510 0x55ed5c17a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 4 client.16704 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed5c0d1a70 con 0x55ed5c178510 2024-04-20T21:42:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.563+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c178510 0x55ed5c17a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.564+0000 7f2cf1400640 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c178510 0x55ed5c17a900 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda61439ce5e567d server_cookie=ac53659efd1e7aea in_seq=0 out_seq=0 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.564+0000 7f2cd3e00640 10 client.16704 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cd3e00640 1 -- 172.21.15.7:0/3457284042 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed5c0d1a70 con 0x55ed5c178510 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cd3e00640 10 client.16704 handle_command_reply: tid=0 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 client.16704 shutdown 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 msgr2=0x55ed5c165670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009fa0 tx=0x7f2ce4019ba0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 shutdown_connections 2024-04-20T21:42:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c1660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c178510 0x55ed5c17a900 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15fa90 0x55ed5c165bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/3457284042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15f170 0x55ed5c165670 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cd3e00640 0 client.16704 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 >> 172.21.15.7:0/3457284042 conn(0x55ed5c1598a0 msgr2=0x55ed5c15b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 shutdown_connections 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 1 -- 172.21.15.7:0/3457284042 wait complete. 2024-04-20T21:42:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.565+0000 7f2cfbc34740 20 client.16704 trim_cache size 0 max 16384 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.580+0000 7f2cfbc34740 1 Processor -- start 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.580+0000 7f2cfbc34740 1 -- start start 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.580+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c167da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.580+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 0x55ed5c1648e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.580+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b6940 con 0x55ed5c168670 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b4200 con 0x55ed5c15fa90 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b2070 con 0x55ed5c15f170 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45942/0 (socket says 172.21.15.7:45942) 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 -- 172.21.15.7:0/399259162 learned_addr learned my addr 172.21.15.7:0/399259162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf1400640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 0x55ed5c1648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 -- 172.21.15.7:0/399259162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 msgr2=0x55ed5c167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 -- 172.21.15.7:0/399259162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 msgr2=0x55ed5c1648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 0x55ed5c1648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf2800640 1 -- 172.21.15.7:0/399259162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed5c15f8c0 con 0x55ed5c168670 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.581+0000 7f2cf1400640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 0x55ed5c1648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.582+0000 7f2cf2800640 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00dae0 tx=0x7f2cec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68266cc4c4c5b160 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.582+0000 7f2cdea00640 1 -- 172.21.15.7:0/399259162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec020040 con 0x55ed5c168670 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.582+0000 7f2cdea00640 1 -- 172.21.15.7:0/399259162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cec004040 con 0x55ed5c168670 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.582+0000 7f2cdea00640 1 -- 172.21.15.7:0/399259162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec005360 con 0x55ed5c168670 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- 172.21.15.7:0/399259162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 msgr2=0x55ed5c174320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00dae0 tx=0x7f2cec016000 comp rx=0 tx=0).stop 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- 172.21.15.7:0/399259162 shutdown_connections 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c15fa90 0x55ed5c1648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c168670 0x55ed5c174320 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/399259162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- 172.21.15.7:0/399259162 >> 172.21.15.7:0/399259162 conn(0x55ed5c1598a0 msgr2=0x55ed5c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- 172.21.15.7:0/399259162 shutdown_connections 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- 172.21.15.7:0/399259162 wait complete. 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.583+0000 7f2cfbc34740 1 -- start start 2024-04-20T21:42:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 Processor -- start 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c16c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b26c0 con 0x55ed5c15fa90 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b3b30 con 0x55ed5c168670 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf41b2d90 con 0x55ed5c15f170 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cfbc34740 10 client.0 _ll_register_callbacks cb 0x7f2ce40197b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cf1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cf1e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c16c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cf1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45944/0 (socket says 172.21.15.7:45944) 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cf1400640 1 -- 172.21.15.7:0/1188920636 learned_addr learned my addr 172.21.15.7:0/1188920636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.584+0000 7f2cf2800640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 -- 172.21.15.7:0/1188920636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 msgr2=0x55ed5c16c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c16c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 -- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 msgr2=0x55ed5c16cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 -- 172.21.15.7:0/1188920636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5bdd63f0 con 0x55ed5c15fa90 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c16c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cf1400640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00efb80 tx=0x7f2ce0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdc931e0d0ea4be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00d7060 con 0x55ed5c15fa90 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f2ce00ef800 con 0x55ed5c15fa90 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.585+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cec004040 con 0x55ed5c15fa90 2024-04-20T21:42:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ce00bdee0 con 0x55ed5c15fa90 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 10 client.16710 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ed5bea07d0 con 0x55ed5c15fa90 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cec004040 con 0x55ed5c15fa90 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f2ce003c030 con 0x55ed5c15fa90 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 10 client.16710 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 20 client.16710 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c1df100 0x55ed5c1e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 4 client.16710 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.586+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed5c0cb8e0 con 0x55ed5c1df100 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.587+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c1df100 0x55ed5c1e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.587+0000 7f2cf1e00640 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c1df100 0x55ed5c1e1520 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23432c9991784e9 server_cookie=a4b57f507c9fb4e9 in_seq=0 out_seq=0 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.587+0000 7f2cd3e00640 10 client.16710 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cd3e00640 1 -- 172.21.15.7:0/1188920636 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed5c0cb8e0 con 0x55ed5c1df100 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cd3e00640 10 client.16710 handle_command_reply: tid=0 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 client.16710 shutdown 2024-04-20T21:42:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 msgr2=0x55ed5c16c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00efb80 tx=0x7f2ce0004610 comp rx=0 tx=0).stop 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 shutdown_connections 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed5c168670 0x55ed5c16cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ed5c1df100 0x55ed5c1e1520 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ed5c15fa90 0x55ed5c16c830 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 --2- 172.21.15.7:0/1188920636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ed5c15f170 0x55ed5c16c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 >> 172.21.15.7:0/1188920636 conn(0x55ed5c1598a0 msgr2=0x55ed5c165610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cd3e00640 0 client.16710 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 shutdown_connections 2024-04-20T21:42:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 1 -- 172.21.15.7:0/1188920636 wait complete. 2024-04-20T21:42:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.588+0000 7f2cfbc34740 20 client.16710 trim_cache size 0 max 16384 2024-04-20T21:42:04.596 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 msgr2=0x7f2cd80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 0x7f2cd80a6260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00e780 tx=0x7f2cec010040 comp rx=0 tx=0).stop 2024-04-20T21:42:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 msgr2=0x7f2cf419f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf419f2c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00057c0 tx=0x7f2ce0004360 comp rx=0 tx=0).stop 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 shutdown_connections 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2cf4112bf0 0x7f2cf41a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2cf4076db0 0x7f2cf419f2c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f2cd80a3da0 0x7f2cd80a6260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 --2- 172.21.15.7:0/3747839059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4076490 0x7f2cf419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 >> 172.21.15.7:0/3747839059 conn(0x7f2cf40ffa00 msgr2=0x7f2cf41143b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 shutdown_connections 2024-04-20T21:42:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:04.589+0000 7f2cf3200640 1 -- 172.21.15.7:0/3747839059 wait complete. 2024-04-20T21:42:04.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:04 smithi007 ceph-mon[16867]: pgmap v269: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:42:04.604 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:04 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3982731619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:04 smithi016 ceph-mon[21409]: pgmap v269: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:42:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:04 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3982731619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:04.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:04 smithi186 ceph-mon[22157]: pgmap v269: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-20T21:42:04.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:04 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3982731619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:05.609 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:42:05.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:42:05.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:42:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.838+0000 7f3233200640 1 -- 172.21.15.7:0/1606145311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234104b90 msgr2=0x7f3234108bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.838+0000 7f3233200640 1 --2- 172.21.15.7:0/1606145311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234104b90 0x7f3234108bd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 -- 172.21.15.7:0/1606145311 shutdown_connections 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 --2- 172.21.15.7:0/1606145311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3234104b90 0x7f3234108bd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 --2- 172.21.15.7:0/1606145311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341092b0 0x7f3234110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 --2- 172.21.15.7:0/1606145311 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341041e0 0x7f32341045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 -- 172.21.15.7:0/1606145311 >> 172.21.15.7:0/1606145311 conn(0x7f32340ffa40 msgr2=0x7f3234101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 -- 172.21.15.7:0/1606145311 shutdown_connections 2024-04-20T21:42:05.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.839+0000 7f3233200640 1 -- 172.21.15.7:0/1606145311 wait complete. 2024-04-20T21:42:05.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 Processor -- start 2024-04-20T21:42:05.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 -- start start 2024-04-20T21:42:05.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341092b0 0x7f32341a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323419a790 0x7f323419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3234114340 con 0x7f32341041e0 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32341141c0 con 0x7f323419a790 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.840+0000 7f3233200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f32341144c0 con 0x7f32341092b0 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45962/0 (socket says 172.21.15.7:45962) 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 -- 172.21.15.7:0/277264742 learned_addr learned my addr 172.21.15.7:0/277264742 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3232800640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323419a790 0x7f323419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231400640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341092b0 0x7f32341a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 -- 172.21.15.7:0/277264742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341092b0 msgr2=0x7f32341a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341092b0 0x7f32341a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 -- 172.21.15.7:0/277264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323419a790 msgr2=0x7f323419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323419a790 0x7f323419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 -- 172.21.15.7:0/277264742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323419b320 con 0x7f32341041e0 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.841+0000 7f3231e00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9f0 tx=0x7f322400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.842+0000 7f322aa00640 1 -- 172.21.15.7:0/277264742 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400cd80 con 0x7f32341041e0 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.842+0000 7f322aa00640 1 -- 172.21.15.7:0/277264742 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224005500 con 0x7f32341041e0 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.842+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323419b610 con 0x7f32341041e0 2024-04-20T21:42:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.842+0000 7f322aa00640 1 -- 172.21.15.7:0/277264742 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224010690 con 0x7f32341041e0 2024-04-20T21:42:05.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.842+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32341a77c0 con 0x7f32341041e0 2024-04-20T21:42:05.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.845+0000 7f323bdd3740 1 Processor -- start 2024-04-20T21:42:05.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 -- start start 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f322aa00640 1 -- 172.21.15.7:0/277264742 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f32240046c0 con 0x7f32341041e0 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55804fb15d20 con 0x55804fbb10f0 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55804fabc3e0 con 0x55804fbba5f0 2024-04-20T21:42:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f323bdd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55804fb11ce0 con 0x55804fbb1a10 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f3232800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f3231400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f3232800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45970/0 (socket says 172.21.15.7:45970) 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f3232800640 1 -- 172.21.15.7:0/807761722 learned_addr learned my addr 172.21.15.7:0/807761722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.846+0000 7f3231e00640 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 -- 172.21.15.7:0/807761722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 msgr2=0x55804fbba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 -- 172.21.15.7:0/807761722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 msgr2=0x55804fbbc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 -- 172.21.15.7:0/807761722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804fadb980 con 0x55804fbb10f0 2024-04-20T21:42:05.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3231400640 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:05.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3232800640 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f322c009f70 tx=0x7f322c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ae79a60cb5183 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3223400640 1 -- 172.21.15.7:0/807761722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c016070 con 0x55804fbb10f0 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3223400640 1 -- 172.21.15.7:0/807761722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322c004d20 con 0x55804fbb10f0 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.847+0000 7f3223400640 1 -- 172.21.15.7:0/807761722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c012950 con 0x55804fbb10f0 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f322aa00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 0x7f32180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f322aa00640 1 -- 172.21.15.7:0/277264742 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f32240d0b70 con 0x7f32341041e0 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f3231e00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 0x7f32180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 -- 172.21.15.7:0/807761722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 msgr2=0x55804fbb14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f322c009f70 tx=0x7f322c010800 comp rx=0 tx=0).stop 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 -- 172.21.15.7:0/807761722 shutdown_connections 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 --2- 172.21.15.7:0/807761722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb14d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.848+0000 7f323bdd3740 1 -- 172.21.15.7:0/807761722 >> 172.21.15.7:0/807761722 conn(0x55804fbab820 msgr2=0x55804fbabc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.849+0000 7f323bdd3740 1 -- 172.21.15.7:0/807761722 shutdown_connections 2024-04-20T21:42:05.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.849+0000 7f323bdd3740 1 -- 172.21.15.7:0/807761722 wait complete. 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.849+0000 7f3231e00640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 0x7f32180a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f32240ce390 tx=0x7f32240d46c0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.849+0000 7f323bdd3740 1 -- start start 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.849+0000 7f323bdd3740 1 Processor -- start 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbb8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55804fbbd3e0 con 0x55804fbb10f0 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55804facf7a0 con 0x55804fbba5f0 2024-04-20T21:42:05.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55804fbbd560 con 0x55804fbb1a10 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f323bdd3740 10 client.0 _ll_register_callbacks cb 0x55804fad1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f3232800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f3232800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45986/0 (socket says 172.21.15.7:45986) 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f3232800640 1 -- 172.21.15.7:0/1255138483 learned_addr learned my addr 172.21.15.7:0/1255138483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f3231400640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.850+0000 7f3232800640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 -- 172.21.15.7:0/1255138483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 msgr2=0x55804fbb7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbb7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 -- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 msgr2=0x55804fbb8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbb8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 -- 172.21.15.7:0/1255138483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804f8f07e0 con 0x55804fbb10f0 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3231400640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3232800640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f322c006010 tx=0x7f322c012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0e380dd8ce2b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3217e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322c016030 con 0x55804fbb10f0 2024-04-20T21:42:05.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f322c003d80 con 0x55804fbb10f0 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.851+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322c004d20 con 0x55804fbb10f0 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f322c031050 con 0x55804fbb10f0 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 10 client.16722 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55804f950f70 con 0x55804fbb10f0 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322c004d20 con 0x55804fbb10f0 2024-04-20T21:42:05.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f322c00f040 con 0x55804fbb10f0 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 10 client.16722 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 20 client.16722 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fbca8c0 0x55804fbcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 4 client.16722 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.852+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55804fb23740 con 0x55804fbca8c0 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.853+0000 7f3231400640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fbca8c0 0x55804fbcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.853+0000 7f3231400640 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fbca8c0 0x55804fbcccb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48f8cd9069477b7 server_cookie=a52a7724f941164e in_seq=0 out_seq=0 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.853+0000 7f3217e00640 10 client.16722 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f3217e00640 1 -- 172.21.15.7:0/1255138483 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55804fb23740 con 0x55804fbca8c0 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f3217e00640 10 client.16722 handle_command_reply: tid=0 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 client.16722 shutdown 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 msgr2=0x55804fbb7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f322c006010 tx=0x7f322c012590 comp rx=0 tx=0).stop 2024-04-20T21:42:05.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 shutdown_connections 2024-04-20T21:42:05.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbb8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fbca8c0 0x55804fbcccb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb1a10 0x55804fbb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 --2- 172.21.15.7:0/1255138483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb10f0 0x55804fbb7680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f3217e00640 0 client.16722 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.854+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 >> 172.21.15.7:0/1255138483 conn(0x55804fbab820 msgr2=0x55804fbad570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.855+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 shutdown_connections 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.855+0000 7f323bdd3740 1 -- 172.21.15.7:0/1255138483 wait complete. 2024-04-20T21:42:05.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.855+0000 7f323bdd3740 20 client.16722 trim_cache size 0 max 16384 2024-04-20T21:42:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.874+0000 7f323bdd3740 1 Processor -- start 2024-04-20T21:42:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.874+0000 7f323bdd3740 1 -- start start 2024-04-20T21:42:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.874+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbd22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f323bdd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32341b2600 con 0x55804fbb1a10 2024-04-20T21:42:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f323bdd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32341afec0 con 0x55804fbba5f0 2024-04-20T21:42:05.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f323bdd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f32341add30 con 0x55804fbb10f0 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46002/0 (socket says 172.21.15.7:46002) 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 -- 172.21.15.7:0/2198445683 learned_addr learned my addr 172.21.15.7:0/2198445683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3232800640 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3232800640 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbd22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 -- 172.21.15.7:0/2198445683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 msgr2=0x55804fbc62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 -- 172.21.15.7:0/2198445683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 msgr2=0x55804fbd22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbd22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.875+0000 7f3231400640 1 -- 172.21.15.7:0/2198445683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804f7e8170 con 0x55804fbb1a10 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.876+0000 7f3231400640 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb507571b15ecad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.876+0000 7f3222a00640 1 -- 172.21.15.7:0/2198445683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c004040 con 0x55804fbb1a10 2024-04-20T21:42:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.876+0000 7f3222a00640 1 -- 172.21.15.7:0/2198445683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c04d070 con 0x55804fbb1a10 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.876+0000 7f3222a00640 1 -- 172.21.15.7:0/2198445683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c005260 con 0x55804fbb1a10 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- 172.21.15.7:0/2198445683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 msgr2=0x55804fbb6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f321c009900 tx=0x7f321c005fd0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- 172.21.15.7:0/2198445683 shutdown_connections 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbd22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbb6e30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2198445683 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- 172.21.15.7:0/2198445683 >> 172.21.15.7:0/2198445683 conn(0x55804fbab820 msgr2=0x55804fbb7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- 172.21.15.7:0/2198445683 shutdown_connections 2024-04-20T21:42:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- 172.21.15.7:0/2198445683 wait complete. 2024-04-20T21:42:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.877+0000 7f323bdd3740 1 -- start start 2024-04-20T21:42:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 Processor -- start 2024-04-20T21:42:05.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbe780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32341ae380 con 0x55804fbb1a10 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32341af7f0 con 0x55804fbba5f0 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f32341aea50 con 0x55804fbb10f0 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f323bdd3740 10 client.0 _ll_register_callbacks cb 0x7f321c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f3231400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f3231400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46018/0 (socket says 172.21.15.7:46018) 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f3231400640 1 -- 172.21.15.7:0/2405792091 learned_addr learned my addr 172.21.15.7:0/2405792091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.878+0000 7f3232800640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3232800640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbe780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 -- 172.21.15.7:0/2405792091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 msgr2=0x55804fbc2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 -- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 msgr2=0x55804fbbe780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbe780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 -- 172.21.15.7:0/2405792091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804fc2fb20 con 0x55804fbb1a10 2024-04-20T21:42:05.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3231400640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f321c02bc70 tx=0x7f321c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53334ea1ce12fcc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3217e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c005850 con 0x55804fbb1a10 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f321c0046c0 con 0x55804fbb1a10 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.879+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f321c04d070 con 0x55804fbb1a10 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f321c05e050 con 0x55804fbb1a10 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 10 client.16728 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55804fc30cf0 con 0x55804fbb1a10 2024-04-20T21:42:05.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c04d070 con 0x55804fbb1a10 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f321c003630 con 0x55804fbb1a10 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 10 client.16728 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 20 client.16728 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fc31490 0x55804fc338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 4 client.16728 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.880+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55804fb1d5b0 con 0x55804fc31490 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.881+0000 7f3232800640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fc31490 0x55804fc338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:05.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.881+0000 7f3232800640 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fc31490 0x55804fc338b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152aea7214607251 server_cookie=942a68db87ed0579 in_seq=0 out_seq=0 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.881+0000 7f3217e00640 10 client.16728 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f3217e00640 1 -- 172.21.15.7:0/2405792091 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55804fb1d5b0 con 0x55804fc31490 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f3217e00640 10 client.16728 handle_command_reply: tid=0 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 client.16728 shutdown 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 msgr2=0x55804fbbe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f321c02bc70 tx=0x7f321c0040f0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 shutdown_connections 2024-04-20T21:42:05.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55804fbba5f0 0x55804fbbe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.882+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55804fc31490 0x55804fc338b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55804fbb1a10 0x55804fbbe040 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 1 --2- 172.21.15.7:0/2405792091 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55804fbb10f0 0x55804fbc2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 >> 172.21.15.7:0/2405792091 conn(0x55804fbab820 msgr2=0x55804fbac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f3217e00640 0 client.16728 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 shutdown_connections 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 1 -- 172.21.15.7:0/2405792091 wait complete. 2024-04-20T21:42:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.883+0000 7f323bdd3740 20 client.16728 trim_cache size 0 max 16384 2024-04-20T21:42:05.891 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:05.891 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:42:05.891 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "60315179-b36f-49a5-8cdb-aa9dd263b50f", 2024-04-20T21:42:05.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:42:05.892 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 msgr2=0x7f32180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 0x7f32180a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f32240ce390 tx=0x7f32240d46c0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 msgr2=0x7f32341a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9f0 tx=0x7f322400edf0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 shutdown_connections 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f32180a3da0 0x7f32180a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323419a790 0x7f323419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32341092b0 0x7f32341a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 --2- 172.21.15.7:0/277264742 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32341041e0 0x7f32341a0620 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 >> 172.21.15.7:0/277264742 conn(0x7f32340ffa40 msgr2=0x7f3234100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 shutdown_connections 2024-04-20T21:42:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:05.884+0000 7f3233200640 1 -- 172.21.15.7:0/277264742 wait complete. 2024-04-20T21:42:05.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:05.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.123+0000 7f1e8e600640 1 -- 172.21.15.7:0/595929669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8810e3e0 msgr2=0x7f1e8810e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 --2- 172.21.15.7:0/595929669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8810e3e0 0x7f1e8810e7c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1e78009900 tx=0x7f1e7802bc30 comp rx=0 tx=0).stop 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 -- 172.21.15.7:0/595929669 shutdown_connections 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 --2- 172.21.15.7:0/595929669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e88108e40 0x7f1e88115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 --2- 172.21.15.7:0/595929669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e881084a0 0x7f1e88108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 --2- 172.21.15.7:0/595929669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8810e3e0 0x7f1e8810e7c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 -- 172.21.15.7:0/595929669 >> 172.21.15.7:0/595929669 conn(0x7f1e88077d70 msgr2=0x7f1e88106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 -- 172.21.15.7:0/595929669 shutdown_connections 2024-04-20T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.124+0000 7f1e8e600640 1 -- 172.21.15.7:0/595929669 wait complete. 2024-04-20T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 Processor -- start 2024-04-20T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 -- start start 2024-04-20T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e88108e40 0x7f1e881a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8810e3e0 0x7f1e8819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e88118740 con 0x7f1e88108e40 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881185c0 con 0x7f1e881084a0 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.125+0000 7f1e8e600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881188c0 con 0x7f1e8810e3e0 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e87e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e88108e40 0x7f1e881a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56550/0 (socket says 172.21.15.7:56550) 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 -- 172.21.15.7:0/312521596 learned_addr learned my addr 172.21.15.7:0/312521596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8810e3e0 0x7f1e8819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 -- 172.21.15.7:0/312521596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8810e3e0 msgr2=0x7f1e8819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8810e3e0 0x7f1e8819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 -- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e88108e40 msgr2=0x7f1e881a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e88108e40 0x7f1e881a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.126+0000 7f1e8d200640 1 -- 172.21.15.7:0/312521596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8819f1e0 con 0x7f1e881084a0 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e8d200640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802efd0 tx=0x7f1e78004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e85600640 1 -- 172.21.15.7:0/312521596 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78041070 con 0x7f1e881084a0 2024-04-20T21:42:06.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e85600640 1 -- 172.21.15.7:0/312521596 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e780054e0 con 0x7f1e881084a0 2024-04-20T21:42:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8819f470 con 0x7f1e881084a0 2024-04-20T21:42:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e85600640 1 -- 172.21.15.7:0/312521596 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e78003d40 con 0x7f1e881084a0 2024-04-20T21:42:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.127+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e881ab770 con 0x7f1e881084a0 2024-04-20T21:42:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.130+0000 7f1e92a35740 1 Processor -- start 2024-04-20T21:42:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.130+0000 7f1e85600640 1 -- 172.21.15.7:0/312521596 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1e78044070 con 0x7f1e881084a0 2024-04-20T21:42:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 -- start start 2024-04-20T21:42:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4f1880 0x55646b4f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55646b44e150 con 0x55646b4e8380 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55646b3f4370 con 0x55646b4e8ca0 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e92a35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55646b44a110 con 0x55646b4f1880 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.131+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56564/0 (socket says 172.21.15.7:56564) 2024-04-20T21:42:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8d200640 1 -- 172.21.15.7:0/3434876045 learned_addr learned my addr 172.21.15.7:0/3434876045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46050/0 (socket says 172.21.15.7:46050) 2024-04-20T21:42:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e87e00640 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4f1880 0x55646b4f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 -- 172.21.15.7:0/3434876045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4f1880 msgr2=0x55646b4f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4f1880 0x55646b4f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 -- 172.21.15.7:0/3434876045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 msgr2=0x55646b4f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8dc00640 1 -- 172.21.15.7:0/3434876045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646b22a7d0 con 0x55646b4e8380 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e8d200640 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:06.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e85600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 0x7f1e6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.132+0000 7f1e85600640 1 -- 172.21.15.7:0/312521596 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1e780f2120 con 0x7f1e881084a0 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.133+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009eb0 tx=0x7f1e800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93048f6dd4edb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.133+0000 7f1e8d200640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 0x7f1e6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.133+0000 7f1e7e000640 1 -- 172.21.15.7:0/3434876045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80003b70 con 0x55646b4e8380 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.133+0000 7f1e7e000640 1 -- 172.21.15.7:0/3434876045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e80015070 con 0x55646b4e8380 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.133+0000 7f1e7e000640 1 -- 172.21.15.7:0/3434876045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8001a920 con 0x55646b4e8380 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- 172.21.15.7:0/3434876045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 msgr2=0x55646b4e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e8d200640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 0x7f1e6c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e780ba700 tx=0x7f1e780f5c70 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009eb0 tx=0x7f1e800107a0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- 172.21.15.7:0/3434876045 shutdown_connections 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4f1880 0x55646b4f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3434876045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4e8760 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- 172.21.15.7:0/3434876045 >> 172.21.15.7:0/3434876045 conn(0x55646b4e2ab0 msgr2=0x55646b4e2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- 172.21.15.7:0/3434876045 shutdown_connections 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- 172.21.15.7:0/3434876045 wait complete. 2024-04-20T21:42:06.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.134+0000 7f1e92a35740 1 -- start start 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 Processor -- start 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b4ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55646b4f4670 con 0x55646b4e8380 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55646b407740 con 0x55646b4f1880 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55646b4f47f0 con 0x55646b4e8ca0 2024-04-20T21:42:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e92a35740 10 client.0 _ll_register_callbacks cb 0x55646b402b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.135+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4ee860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46052/0 (socket says 172.21.15.7:46052) 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e8dc00640 1 -- 172.21.15.7:0/3468105746 learned_addr learned my addr 172.21.15.7:0/3468105746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b4ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 -- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 msgr2=0x55646b4ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b4ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 -- 172.21.15.7:0/3468105746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 msgr2=0x55646b4ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 -- 172.21.15.7:0/3468105746 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646b1603f0 con 0x55646b4e8ca0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e87e00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4eeda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1e881a0250 tx=0x7f1e74012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77285f219a5b66a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.136+0000 7f1e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400f920 con 0x55646b4e8ca0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1e7400fac0 con 0x55646b4e8ca0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e80015070 con 0x55646b4e8ca0 2024-04-20T21:42:06.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e7400fcc0 con 0x55646b4e8ca0 2024-04-20T21:42:06.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e92a35740 10 client.26902 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:06.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55646b1200f0 con 0x55646b4e8ca0 2024-04-20T21:42:06.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.137+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e80015070 con 0x55646b4e8ca0 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1e74023070 con 0x55646b4e8ca0 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 10 client.26902 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 20 client.26902 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b501c40 0x55646b5040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 4 client.26902 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55646b45bb70 con 0x55646b501c40 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.138+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b501c40 0x55646b5040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.139+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b501c40 0x55646b5040c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed0f5b076231707 server_cookie=1dc624647fe652be in_seq=0 out_seq=0 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.139+0000 7f1e72a00640 10 client.26902 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e72a00640 1 -- 172.21.15.7:0/3468105746 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55646b45bb70 con 0x55646b501c40 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e72a00640 10 client.26902 handle_command_reply: tid=0 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 client.26902 shutdown 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 msgr2=0x55646b4eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4eeda0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1e881a0250 tx=0x7f1e74012b60 comp rx=0 tx=0).stop 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 shutdown_connections 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b4ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b501c40 0x55646b5040c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4eeda0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 --2- 172.21.15.7:0/3468105746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e72a00640 0 client.26902 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 >> 172.21.15.7:0/3468105746 conn(0x55646b4e2ab0 msgr2=0x55646b4e4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 shutdown_connections 2024-04-20T21:42:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 1 -- 172.21.15.7:0/3468105746 wait complete. 2024-04-20T21:42:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.140+0000 7f1e92a35740 20 client.26902 trim_cache size 0 max 16384 2024-04-20T21:42:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 Processor -- start 2024-04-20T21:42:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 -- start start 2024-04-20T21:42:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8380 0x55646b4f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4f1880 0x55646b4e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b6990 con 0x55646b4f1880 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b4250 con 0x55646b4e8ca0 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.160+0000 7f1e92a35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b20c0 con 0x55646b4e8380 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:56590/0 (socket says 172.21.15.7:56590) 2024-04-20T21:42:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 -- 172.21.15.7:0/694794320 learned_addr learned my addr 172.21.15.7:0/694794320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 -- 172.21.15.7:0/694794320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8380 msgr2=0x55646b4f0f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8380 0x55646b4f0f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 -- 172.21.15.7:0/694794320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4f1880 msgr2=0x55646b4e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4f1880 0x55646b4e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 -- 172.21.15.7:0/694794320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646b4ecdb0 con 0x55646b4e8ca0 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.161+0000 7f1e87e00640 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e00 tx=0x7f1e740057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd7e3279da51d4d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.162+0000 7f1e7d600640 1 -- 172.21.15.7:0/694794320 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74029070 con 0x55646b4e8ca0 2024-04-20T21:42:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.162+0000 7f1e7d600640 1 -- 172.21.15.7:0/694794320 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7400fe60 con 0x55646b4e8ca0 2024-04-20T21:42:06.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.162+0000 7f1e7d600640 1 -- 172.21.15.7:0/694794320 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7401cbe0 con 0x55646b4e8ca0 2024-04-20T21:42:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- 172.21.15.7:0/694794320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 msgr2=0x55646b4fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e00 tx=0x7f1e740057d0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- 172.21.15.7:0/694794320 shutdown_connections 2024-04-20T21:42:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4e8ca0 0x55646b4fd560 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4f1880 0x55646b4e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 --2- 172.21.15.7:0/694794320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8380 0x55646b4f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- 172.21.15.7:0/694794320 >> 172.21.15.7:0/694794320 conn(0x55646b4e2ab0 msgr2=0x55646b4e3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- 172.21.15.7:0/694794320 shutdown_connections 2024-04-20T21:42:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- 172.21.15.7:0/694794320 wait complete. 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 -- start start 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.163+0000 7f1e92a35740 1 Processor -- start 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b567200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b2710 con 0x55646b4e8380 2024-04-20T21:42:06.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b3b80 con 0x55646b4f1880 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1e881b2de0 con 0x55646b4e8ca0 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e92a35740 10 client.0 _ll_register_callbacks cb 0x7f1e74009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e8dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46074/0 (socket says 172.21.15.7:46074) 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e8dc00640 1 -- 172.21.15.7:0/1527334341 learned_addr learned my addr 172.21.15.7:0/1527334341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e87e00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.164+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b567200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 -- 172.21.15.7:0/1527334341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 msgr2=0x55646b4ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 -- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 msgr2=0x55646b567200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b567200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 -- 172.21.15.7:0/1527334341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646b567740 con 0x55646b4e8380 2024-04-20T21:42:06.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e87e00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e8dc00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e80010ca0 tx=0x7f1e80013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb38ff4d6921d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80025070 con 0x55646b4e8380 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1e8001a810 con 0x55646b4e8380 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7400fe60 con 0x55646b4e8380 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1e8001aa10 con 0x55646b4e8380 2024-04-20T21:42:06.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e92a35740 10 client.16743 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.165+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55646b568950 con 0x55646b4e8380 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e7400fe60 con 0x55646b4e8380 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f1e80035070 con 0x55646b4e8380 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 10 client.16743 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 20 client.16743 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b568ee0 0x55646b56b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:06.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 4 client.16743 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55646b4559e0 con 0x55646b568ee0 2024-04-20T21:42:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.166+0000 7f1e87e00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b568ee0 0x55646b56b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e87e00640 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b568ee0 0x55646b56b2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12604f5e98ab822 server_cookie=d81b0a5ece230a9f in_seq=0 out_seq=0 2024-04-20T21:42:06.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e72a00640 10 client.16743 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e72a00640 1 -- 172.21.15.7:0/1527334341 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55646b4559e0 con 0x55646b568ee0 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e72a00640 10 client.16743 handle_command_reply: tid=0 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 client.16743 shutdown 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 msgr2=0x55646b4eb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e80010ca0 tx=0x7f1e80013040 comp rx=0 tx=0).stop 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 shutdown_connections 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55646b4f1880 0x55646b567200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55646b568ee0 0x55646b56b2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55646b4e8ca0 0x55646b4ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 --2- 172.21.15.7:0/1527334341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55646b4e8380 0x55646b4eb910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e72a00640 0 client.16743 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 >> 172.21.15.7:0/1527334341 conn(0x55646b4e2ab0 msgr2=0x55646b4ef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 shutdown_connections 2024-04-20T21:42:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.167+0000 7f1e92a35740 1 -- 172.21.15.7:0/1527334341 wait complete. 2024-04-20T21:42:06.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e92a35740 20 client.16743 trim_cache size 0 max 16384 2024-04-20T21:42:06.180 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:06.180 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: "60315179-b36f-49a5-8cdb-aa9dd263b50f": { 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "60315179-b36f-49a5-8cdb-aa9dd263b50f", 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:42:06.181 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 msgr2=0x7f1e6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 0x7f1e6c0a6260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1e780ba700 tx=0x7f1e780f5c70 comp rx=0 tx=0).stop 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 msgr2=0x7f1e881a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e7802efd0 tx=0x7f1e78004430 comp rx=0 tx=0).stop 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 shutdown_connections 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e8810e3e0 0x7f1e8819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e88108e40 0x7f1e881a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1e6c0a3da0 0x7f1e6c0a6260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 --2- 172.21.15.7:0/312521596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e881084a0 0x7f1e881a48b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:06.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.168+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 >> 172.21.15.7:0/312521596 conn(0x7f1e88077d70 msgr2=0x7f1e88078e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:06.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.169+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 shutdown_connections 2024-04-20T21:42:06.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:06.169+0000 7f1e8e600640 1 -- 172.21.15.7:0/312521596 wait complete. 2024-04-20T21:42:06.187 INFO:tasks.cephfs.filesystem:scrub status for tag:60315179-b36f-49a5-8cdb-aa9dd263b50f - {'path': '/', 'tag': '60315179-b36f-49a5-8cdb-aa9dd263b50f', 'options': 'recursive,force'} 2024-04-20T21:42:06.767 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: pgmap v270: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:42:06.767 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:06.767 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:06.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:06.768 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:06 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: pgmap v270: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:42:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:06.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:06 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: pgmap v270: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:42:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:06.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:06.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:06.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:06 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:07 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:42:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:07 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:07 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:07 smithi007 ceph-mon[16867]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:42:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:07 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:42:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:07 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:07 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:07 smithi016 ceph-mon[21409]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:42:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:07 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:42:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:07 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:07.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:07 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:07.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:07 smithi186 ceph-mon[22157]: osdmap e94: 12 total, 12 up, 12 in 2024-04-20T21:42:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:08 smithi007 ceph-mon[16867]: pgmap v272: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:42:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:08 smithi016 ceph-mon[21409]: pgmap v272: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:42:08.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:08 smithi186 ceph-mon[22157]: pgmap v272: 97 pgs: 97 active+clean; 21 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-20T21:42:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:10 smithi007 ceph-mon[16867]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:10.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:10 smithi016 ceph-mon[21409]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:10.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:10 smithi186 ceph-mon[22157]: pgmap v273: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:12.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:12 smithi007 ceph-mon[16867]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:12.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:12 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:12.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:12 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:12.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:12 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:12.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:12 smithi016 ceph-mon[21409]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:12.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:12 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:12.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:12 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:12.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:12 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:12 smithi186 ceph-mon[22157]: pgmap v274: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:12 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:12.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:12 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:12.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:12 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:14 smithi007 ceph-mon[16867]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:14 smithi007 ceph-mon[16867]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:42:14.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:14 smithi016 ceph-mon[21409]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:14.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:14 smithi016 ceph-mon[21409]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:42:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:14 smithi186 ceph-mon[22157]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 59 KiB/s wr, 27 op/s 2024-04-20T21:42:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:14 smithi186 ceph-mon[22157]: osdmap e95: 12 total, 12 up, 12 in 2024-04-20T21:42:16.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:16 smithi016 ceph-mon[21409]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-20T21:42:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:16 smithi186 ceph-mon[22157]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-20T21:42:17.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:16 smithi007 ceph-mon[16867]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.3 KiB/s wr, 7 op/s 2024-04-20T21:42:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:18 smithi016 ceph-mon[21409]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-04-20T21:42:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:18 smithi186 ceph-mon[22157]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-04-20T21:42:19.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:18 smithi007 ceph-mon[16867]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.6 KiB/s wr, 6 op/s 2024-04-20T21:42:20.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:20 smithi016 ceph-mon[21409]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:20.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:20 smithi186 ceph-mon[22157]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:21.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:20 smithi007 ceph-mon[16867]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:22.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:22 smithi016 ceph-mon[21409]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:22.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:22 smithi186 ceph-mon[22157]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:23.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:22 smithi007 ceph-mon[16867]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-20T21:42:24.836 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:24 smithi007 ceph-mon[16867]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:42:24.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:24 smithi016 ceph-mon[21409]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:42:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:24 smithi186 ceph-mon[22157]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-20T21:42:26.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:26 smithi016 ceph-mon[21409]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:42:26.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:26 smithi186 ceph-mon[22157]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:42:27.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:26 smithi007 ceph-mon[16867]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-20T21:42:28.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:28 smithi016 ceph-mon[21409]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:28.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:28 smithi186 ceph-mon[22157]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:29.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:28 smithi007 ceph-mon[16867]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:30.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:30 smithi016 ceph-mon[21409]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:30 smithi186 ceph-mon[22157]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:31.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:30 smithi007 ceph-mon[16867]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:32 smithi016 ceph-mon[21409]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:32.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:32 smithi186 ceph-mon[22157]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:33.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:32 smithi007 ceph-mon[16867]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:34 smithi016 ceph-mon[21409]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:42:34.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:34 smithi186 ceph-mon[22157]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:42:35.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:34 smithi007 ceph-mon[16867]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-20T21:42:36.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:36.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:36.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 -- 172.21.15.7:0/417095986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b0074310 msgr2=0x7f40b00746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 --2- 172.21.15.7:0/417095986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b0074310 0x7f40b00746f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ca0 tx=0x7f40a002c180 comp rx=0 tx=0).stop 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 -- 172.21.15.7:0/417095986 shutdown_connections 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 --2- 172.21.15.7:0/417095986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074c30 0x7f40b010e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 --2- 172.21.15.7:0/417095986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b010e620 0x7f40b0110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 --2- 172.21.15.7:0/417095986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b0074310 0x7f40b00746f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 -- 172.21.15.7:0/417095986 >> 172.21.15.7:0/417095986 conn(0x7f40b00fd8b0 msgr2=0x7f40b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 -- 172.21.15.7:0/417095986 shutdown_connections 2024-04-20T21:42:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.431+0000 7f40b7000640 1 -- 172.21.15.7:0/417095986 wait complete. 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.432+0000 7f40b7000640 1 Processor -- start 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 -- start start 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074310 0x7f40b01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b0074c30 0x7f40b01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0114330 con 0x7f40b010e620 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01141b0 con 0x7f40b0074310 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b7000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01144b0 con 0x7f40b0074c30 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45344/0 (socket says 172.21.15.7:45344) 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b6600640 1 -- 172.21.15.7:0/760538068 learned_addr learned my addr 172.21.15.7:0/760538068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b5200640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b0074c30 0x7f40b01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.433+0000 7f40b5c00640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074310 0x7f40b01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 -- 172.21.15.7:0/760538068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b0074c30 msgr2=0x7f40b01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b0074c30 0x7f40b01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 -- 172.21.15.7:0/760538068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074310 msgr2=0x7f40b01a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074310 0x7f40b01a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 -- 172.21.15.7:0/760538068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b019af50 con 0x7f40b010e620 2024-04-20T21:42:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b6600640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e970 tx=0x7f40a800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40ae000640 1 -- 172.21.15.7:0/760538068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800ccc0 con 0x7f40b010e620 2024-04-20T21:42:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40ae000640 1 -- 172.21.15.7:0/760538068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a800ce60 con 0x7f40b010e620 2024-04-20T21:42:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b019b240 con 0x7f40b010e620 2024-04-20T21:42:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40ae000640 1 -- 172.21.15.7:0/760538068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8010490 con 0x7f40b010e620 2024-04-20T21:42:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.434+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b01a7550 con 0x7f40b010e620 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.438+0000 7f40bb3cd740 1 Processor -- start 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.438+0000 7f40ae000640 1 -- 172.21.15.7:0/760538068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f40a8005020 con 0x7f40b010e620 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.438+0000 7f40bb3cd740 1 -- start start 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.438+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56431db82150 con 0x56431dc1c380 2024-04-20T21:42:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56431db28370 con 0x56431dc25880 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56431db7e110 con 0x56431dc1cca0 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45356/0 (socket says 172.21.15.7:45356) 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 -- 172.21.15.7:0/1097693698 learned_addr learned my addr 172.21.15.7:0/1097693698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5c00640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 -- 172.21.15.7:0/1097693698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 msgr2=0x56431dc25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc25340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 -- 172.21.15.7:0/1097693698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 msgr2=0x56431dc27c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc27c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5200640 1 -- 172.21.15.7:0/1097693698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431d95e7d0 con 0x56431dc1c380 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.439+0000 7f40b5c00640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40b5200640 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f409c013620 tx=0x7f409c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e117f9f055e97a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40ae000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 0x7f40940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40a6a00640 1 -- 172.21.15.7:0/1097693698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c004050 con 0x56431dc1c380 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40a6a00640 1 -- 172.21.15.7:0/1097693698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c0041f0 con 0x56431dc1c380 2024-04-20T21:42:36.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40a6a00640 1 -- 172.21.15.7:0/1097693698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00f900 con 0x56431dc1c380 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40ae000640 1 -- 172.21.15.7:0/760538068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f40a80d0bd0 con 0x7f40b010e620 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.440+0000 7f40b5c00640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 0x7f40940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1097693698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 msgr2=0x56431dc1c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f409c013620 tx=0x7f409c009850 comp rx=0 tx=0).stop 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1097693698 shutdown_connections 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc27c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1097693698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc1c760 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1097693698 >> 172.21.15.7:0/1097693698 conn(0x56431dc16ab0 msgr2=0x56431dc16e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40b5c00640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 0x7f40940a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40a0004af0 tx=0x7f40a0005c50 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1097693698 shutdown_connections 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.441+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1097693698 wait complete. 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 -- start start 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 Processor -- start 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 0x56431dc22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 0x56431dc22db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 0x56431dc232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56431dc28610 con 0x56431dc25880 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56431db3b740 con 0x56431dc1cca0 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.442+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56431dc28790 con 0x56431dc1c380 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40bb3cd740 10 client.0 _ll_register_callbacks cb 0x56431db36b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 0x56431dc22db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 0x56431dc232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 0x56431dc22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 0x56431dc22db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42736/0 (socket says 172.21.15.7:42736) 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5200640 1 -- 172.21.15.7:0/3849781662 learned_addr learned my addr 172.21.15.7:0/3849781662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 -- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 msgr2=0x56431dc22db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 0x56431dc22db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 -- 172.21.15.7:0/3849781662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 msgr2=0x56431dc232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 0x56431dc232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b5c00640 1 -- 172.21.15.7:0/3849781662 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431d8943f0 con 0x56431dc1c380 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.443+0000 7f40b6600640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 0x56431dc232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f40b5c00640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 0x56431dc22870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006010 tx=0x7f40a00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ea4f53d254fc84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f409b400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0050020 con 0x56431dc1c380 2024-04-20T21:42:36.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f40a00047a0 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409c0041f0 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40a005b050 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f40bb3cd740 10 client.26917 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.444+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56431d8540f0 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c0041f0 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f40a004c880 con 0x56431dc1c380 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 10 client.26917 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:36.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 20 client.26917 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:36.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dc35ba0 0x56431dc37ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 4 client.26917 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:36.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.445+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56431db8fb70 con 0x56431dc35ba0 2024-04-20T21:42:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.446+0000 7f40b5200640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dc35ba0 0x56431dc37ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.446+0000 7f40b5200640 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dc35ba0 0x56431dc37ff0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38d1f479dd1f71 server_cookie=aaad61068ec71ba4 in_seq=0 out_seq=0 2024-04-20T21:42:36.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.446+0000 7f409b400640 10 client.26917 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.447+0000 7f409b400640 1 -- 172.21.15.7:0/3849781662 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56431db8fb70 con 0x56431dc35ba0 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.447+0000 7f409b400640 10 client.26917 handle_command_reply: tid=0 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.447+0000 7f40bb3cd740 1 client.26917 shutdown 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.447+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 msgr2=0x56431dc22870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.447+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 0x56431dc22870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006010 tx=0x7f40a00095c0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 shutdown_connections 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc1cca0 0x56431dc22db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dc35ba0 0x56431dc37ff0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc25880 0x56431dc232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/3849781662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1c380 0x56431dc22870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f409b400640 0 client.26917 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 >> 172.21.15.7:0/3849781662 conn(0x56431dc16ab0 msgr2=0x56431dc18800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 shutdown_connections 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 1 -- 172.21.15.7:0/3849781662 wait complete. 2024-04-20T21:42:36.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.448+0000 7f40bb3cd740 20 client.26917 trim_cache size 0 max 16384 2024-04-20T21:42:36.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.467+0000 7f40bb3cd740 1 Processor -- start 2024-04-20T21:42:36.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 -- start start 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b2570 con 0x56431dc1c380 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01afe30 con 0x56431dc25880 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01adca0 con 0x56431dc1cca0 2024-04-20T21:42:36.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45368/0 (socket says 172.21.15.7:45368) 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40b5c00640 1 -- 172.21.15.7:0/1636298090 learned_addr learned my addr 172.21.15.7:0/1636298090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40b5200640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.468+0000 7f40b6600640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 -- 172.21.15.7:0/1636298090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 msgr2=0x56431dc314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 -- 172.21.15.7:0/1636298090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 msgr2=0x56431dc3d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc3d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 -- 172.21.15.7:0/1636298090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431dc20dc0 con 0x56431dc1c380 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5200640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40b5c00640 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f40a004cd20 tx=0x7f40a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc2717b06482346 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40a6000640 1 -- 172.21.15.7:0/1636298090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0050070 con 0x56431dc1c380 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40a6000640 1 -- 172.21.15.7:0/1636298090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a004c950 con 0x56431dc1c380 2024-04-20T21:42:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.469+0000 7f40a6000640 1 -- 172.21.15.7:0/1636298090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0039040 con 0x56431dc1c380 2024-04-20T21:42:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1636298090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 msgr2=0x56431dc24fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f40a004cd20 tx=0x7f40a0009b60 comp rx=0 tx=0).stop 2024-04-20T21:42:36.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1636298090 shutdown_connections 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc3d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/1636298090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc24fa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1636298090 >> 172.21.15.7:0/1636298090 conn(0x56431dc16ab0 msgr2=0x56431dc16e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1636298090 shutdown_connections 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.470+0000 7f40bb3cd740 1 -- 172.21.15.7:0/1636298090 wait complete. 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.471+0000 7f40bb3cd740 1 -- start start 2024-04-20T21:42:36.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.471+0000 7f40bb3cd740 1 Processor -- start 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.471+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.471+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.471+0000 7f40bb3cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ae2f0 con 0x56431dc1c380 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01af760 con 0x56431dc25880 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40bb3cd740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ae9c0 con 0x56431dc1cca0 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40bb3cd740 10 client.0 _ll_register_callbacks cb 0x7f40a0044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45378/0 (socket says 172.21.15.7:45378) 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 -- 172.21.15.7:0/2078625763 learned_addr learned my addr 172.21.15.7:0/2078625763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b6600640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5200640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 -- 172.21.15.7:0/2078625763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 msgr2=0x56431dc290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 -- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 msgr2=0x56431dc29630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc29630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b5c00640 1 -- 172.21.15.7:0/2078625763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431dc29e60 con 0x56431dc1c380 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.472+0000 7f40b6600640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f40b5c00640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f40a004cca0 tx=0x7f40a002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e205d4485153eeca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f409b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a006e060 con 0x56431dc1c380 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f40a0032b80 con 0x56431dc1c380 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a004c950 con 0x56431dc1c380 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40a0050040 con 0x56431dc1c380 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f40bb3cd740 10 client.16761 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:36.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.473+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56431dca4940 con 0x56431dc1c380 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a004c950 con 0x56431dc1c380 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f40a0068070 con 0x56431dc1c380 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 10 client.16761 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 20 client.16761 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dca4f50 0x56431dca7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 4 client.16761 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56431db899e0 con 0x56431dca4f50 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.474+0000 7f40b5200640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dca4f50 0x56431dca7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40b5200640 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dca4f50 0x56431dca7340 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5106ac99d683b3 server_cookie=79a2c306f2daba09 in_seq=0 out_seq=0 2024-04-20T21:42:36.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f409b400640 10 client.16761 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:36.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f409b400640 1 -- 172.21.15.7:0/2078625763 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56431db899e0 con 0x56431dca4f50 2024-04-20T21:42:36.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f409b400640 10 client.16761 handle_command_reply: tid=0 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 client.16761 shutdown 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 msgr2=0x56431dc2dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f40a004cca0 tx=0x7f40a002c880 comp rx=0 tx=0).stop 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 shutdown_connections 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56431dc25880 0x56431dc29630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56431dca4f50 0x56431dca7340 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431dc1cca0 0x56431dc290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 --2- 172.21.15.7:0/2078625763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56431dc1c380 0x56431dc2dd80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f409b400640 0 client.16761 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 >> 172.21.15.7:0/2078625763 conn(0x56431dc16ab0 msgr2=0x56431dc22810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 shutdown_connections 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 1 -- 172.21.15.7:0/2078625763 wait complete. 2024-04-20T21:42:36.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.475+0000 7f40bb3cd740 20 client.16761 trim_cache size 0 max 16384 2024-04-20T21:42:36.484 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:36.484 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:42:36.484 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:42:36.484 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 msgr2=0x7f40940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 0x7f40940a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f40a0004af0 tx=0x7f40a0005c50 comp rx=0 tx=0).stop 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 msgr2=0x7f40b019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e970 tx=0x7f40a800ed70 comp rx=0 tx=0).stop 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 shutdown_connections 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b010e620 0x7f40b019a690 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b0074c30 0x7f40b01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f40940a3d20 0x7f40940a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 --2- 172.21.15.7:0/760538068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40b0074310 0x7f40b01a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 >> 172.21.15.7:0/760538068 conn(0x7f40b00fd8b0 msgr2=0x7f40b00fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 shutdown_connections 2024-04-20T21:42:36.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.476+0000 7f40b7000640 1 -- 172.21.15.7:0/760538068 wait complete. 2024-04-20T21:42:36.493 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:42:36.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:42:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 -- 172.21.15.7:0/2457528766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60100dd0 msgr2=0x7f5e601011b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 --2- 172.21.15.7:0/2457528766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60100dd0 0x7f5e601011b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009860 tx=0x7f5e5402bc10 comp rx=0 tx=0).stop 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 -- 172.21.15.7:0/2457528766 shutdown_connections 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 --2- 172.21.15.7:0/2457528766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e60106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 --2- 172.21.15.7:0/2457528766 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e601016f0 0x7f5e60105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 --2- 172.21.15.7:0/2457528766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e60100dd0 0x7f5e601011b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.685+0000 7f5e67000640 1 -- 172.21.15.7:0/2457528766 >> 172.21.15.7:0/2457528766 conn(0x7f5e600fc7e0 msgr2=0x7f5e600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.686+0000 7f5e67000640 1 -- 172.21.15.7:0/2457528766 shutdown_connections 2024-04-20T21:42:36.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.686+0000 7f5e67000640 1 -- 172.21.15.7:0/2457528766 wait complete. 2024-04-20T21:42:36.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.687+0000 7f5e67000640 1 Processor -- start 2024-04-20T21:42:36.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.687+0000 7f5e67000640 1 -- start start 2024-04-20T21:42:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 0x7f5e6019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e601016f0 0x7f5e6019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e601187c0 con 0x7f5e601061d0 2024-04-20T21:42:36.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60118640 con 0x7f5e601016f0 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e67000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60118940 con 0x7f5e60100dd0 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45410/0 (socket says 172.21.15.7:45410) 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 -- 172.21.15.7:0/1605791986 learned_addr learned my addr 172.21.15.7:0/1605791986 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e65c00640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 0x7f5e6019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e65200640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e601016f0 0x7f5e6019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 -- 172.21.15.7:0/1605791986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 msgr2=0x7f5e6019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 0x7f5e6019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 -- 172.21.15.7:0/1605791986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e601016f0 msgr2=0x7f5e6019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.688+0000 7f5e66600640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e601016f0 0x7f5e6019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e66600640 1 -- 172.21.15.7:0/1605791986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601a3db0 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e65c00640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 0x7f5e6019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e66600640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800efc0 tx=0x7f5e5800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58003970 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601a40a0 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e58004020 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800b9f0 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.689+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601ab930 con 0x7f5e601061d0 2024-04-20T21:42:36.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.690+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6004fd60 con 0x7f5e601061d0 2024-04-20T21:42:36.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.692+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5e5801d020 con 0x7f5e601061d0 2024-04-20T21:42:36.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.695+0000 7f5e5e000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 0x7f5e480a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.695+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f5e580d0640 con 0x7f5e601061d0 2024-04-20T21:42:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.695+0000 7f5e65c00640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 0x7f5e480a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:36.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.696+0000 7f5e65c00640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 0x7f5e480a61d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009830 tx=0x7f5e5402e040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:36.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.700+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f5e5809cca0 con 0x7f5e601061d0 2024-04-20T21:42:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.942+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e601abb00 con 0x7f5e601061d0 2024-04-20T21:42:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.944+0000 7f5e5e000640 1 -- 172.21.15.7:0/1605791986 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f5e5809c5a0 con 0x7f5e601061d0 2024-04-20T21:42:36.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:42:36.943 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:42:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:42:36.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:36 smithi016 ceph-mon[21409]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:42:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 msgr2=0x7f5e480a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 0x7f5e480a61d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009830 tx=0x7f5e5402e040 comp rx=0 tx=0).stop 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 msgr2=0x7f5e601a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800efc0 tx=0x7f5e5800c800 comp rx=0 tx=0).stop 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 shutdown_connections 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e480a3d10 0x7f5e480a61d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e601016f0 0x7f5e6019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e601061d0 0x7f5e601a3680 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 --2- 172.21.15.7:0/1605791986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5e60100dd0 0x7f5e6019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.946+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 >> 172.21.15.7:0/1605791986 conn(0x7f5e600fc7e0 msgr2=0x7f5e600fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.947+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 shutdown_connections 2024-04-20T21:42:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:36.947+0000 7f5e67000640 1 -- 172.21.15.7:0/1605791986 wait complete. 2024-04-20T21:42:36.962 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:36 smithi007 ceph-mon[16867]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:42:36.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:42:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:36 smithi186 ceph-mon[22157]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-20T21:42:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.180+0000 7f3fc8200640 1 -- 172.21.15.7:0/3890040492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0107260 msgr2=0x7f3fc010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.180+0000 7f3fc8200640 1 --2- 172.21.15.7:0/3890040492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0107260 0x7f3fc010ee30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009860 tx=0x7f3fb802bb70 comp rx=0 tx=0).stop 2024-04-20T21:42:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 -- 172.21.15.7:0/3890040492 shutdown_connections 2024-04-20T21:42:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 --2- 172.21.15.7:0/3890040492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0107260 0x7f3fc010ee30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 --2- 172.21.15.7:0/3890040492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0102b40 0x7f3fc0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 --2- 172.21.15.7:0/3890040492 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc0102190 0x7f3fc0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 -- 172.21.15.7:0/3890040492 >> 172.21.15.7:0/3890040492 conn(0x7f3fc00fda10 msgr2=0x7f3fc00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 -- 172.21.15.7:0/3890040492 shutdown_connections 2024-04-20T21:42:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.181+0000 7f3fc8200640 1 -- 172.21.15.7:0/3890040492 wait complete. 2024-04-20T21:42:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.182+0000 7f3fc8200640 1 Processor -- start 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.182+0000 7f3fc8200640 1 -- start start 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.182+0000 7f3fc8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc8200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0107260 0x7f3fc01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc8200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc019a810 0x7f3fc019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc8200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0112300 con 0x7f3fc0107260 2024-04-20T21:42:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0112180 con 0x7f3fc0102190 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc8200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc0112480 con 0x7f3fc019a810 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0107260 0x7f3fc01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42810/0 (socket says 172.21.15.7:42810) 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 -- 172.21.15.7:0/471803065 learned_addr learned my addr 172.21.15.7:0/471803065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc6200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc019a810 0x7f3fc019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 -- 172.21.15.7:0/471803065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc019a810 msgr2=0x7f3fc019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc019a810 0x7f3fc019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 -- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0107260 msgr2=0x7f3fc01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0107260 0x7f3fc01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.183+0000 7f3fc5800640 1 -- 172.21.15.7:0/471803065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc019b3a0 con 0x7f3fc0102190 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fc5800640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000e9f0 tx=0x7f3fb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fbe000640 1 -- 172.21.15.7:0/471803065 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000cd80 con 0x7f3fc0102190 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fbe000640 1 -- 172.21.15.7:0/471803065 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb0010910 con 0x7f3fc0102190 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc019b690 con 0x7f3fc0102190 2024-04-20T21:42:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fbe000640 1 -- 172.21.15.7:0/471803065 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0018990 con 0x7f3fc0102190 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.184+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01a78c0 con 0x7f3fc0102190 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.187+0000 7f3fcaf9b740 1 Processor -- start 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.187+0000 7f3fcaf9b740 1 -- start start 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fbe000640 1 -- 172.21.15.7:0/471803065 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3fb0010ac0 con 0x7f3fc0102190 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b98d0 0x557c6b1c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557c6b11e050 con 0x557c6b1b98d0 2024-04-20T21:42:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c6b0c4490 con 0x557c6b1c24b0 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557c6b11a010 con 0x557c6b1b8fb0 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58176/0 (socket says 172.21.15.7:58176) 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc6200640 1 -- 172.21.15.7:0/3340427587 learned_addr learned my addr 172.21.15.7:0/3340427587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc5800640 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b98d0 0x557c6b1c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc6200640 1 -- 172.21.15.7:0/3340427587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 msgr2=0x557c6b1c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.188+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fc6200640 1 -- 172.21.15.7:0/3340427587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b98d0 msgr2=0x557c6b1c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b98d0 0x557c6b1c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fc6200640 1 -- 172.21.15.7:0/3340427587 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6b0cc070 con 0x557c6b1b8fb0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8037040 tx=0x7f3fb8009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=274e5a52f4df409c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fb6a00640 1 -- 172.21.15.7:0/3340427587 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8043030 con 0x557c6b1b8fb0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fb6a00640 1 -- 172.21.15.7:0/3340427587 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8004040 con 0x557c6b1b8fb0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fb6a00640 1 -- 172.21.15.7:0/3340427587 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8039040 con 0x557c6b1b8fb0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.189+0000 7f3fbe000640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 0x7f3fa40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fbe000640 1 -- 172.21.15.7:0/471803065 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3fb00d0ac0 con 0x7f3fc0102190 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fc5800640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 0x7f3fa40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3340427587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 msgr2=0x557c6b1b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8037040 tx=0x7f3fb8009490 comp rx=0 tx=0).stop 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3340427587 shutdown_connections 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b98d0 0x557c6b1c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3340427587 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b8fb0 0x557c6b1b9390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3340427587 >> 172.21.15.7:0/3340427587 conn(0x557c6b1b36e0 msgr2=0x557c6b1b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3340427587 shutdown_connections 2024-04-20T21:42:37.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.190+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3340427587 wait complete. 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.191+0000 7f3fc5800640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 0x7f3fa40a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x557c6b1b8a80 tx=0x7f3fb00d4570 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.191+0000 7f3fcaf9b740 1 -- start start 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.191+0000 7f3fcaf9b740 1 Processor -- start 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.191+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.191+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557c6b1c5240 con 0x557c6b1b8fb0 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557c6b0d7860 con 0x557c6b1b98d0 2024-04-20T21:42:37.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557c6b1c53c0 con 0x557c6b1c24b0 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fcaf9b740 10 client.0 _ll_register_callbacks cb 0x557c6b0a5fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42814/0 (socket says 172.21.15.7:42814) 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 -- 172.21.15.7:0/4031892871 learned_addr learned my addr 172.21.15.7:0/4031892871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45458/0 (socket says 172.21.15.7:45458) 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc6200640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 -- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 msgr2=0x557c6b1bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 -- 172.21.15.7:0/4031892871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 msgr2=0x557c6b1bf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc4e00640 1 -- 172.21.15.7:0/4031892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6b0db950 con 0x557c6b1b98d0 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.192+0000 7f3fc6200640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fac005fd0 tx=0x7f3fac00d7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc17606b95430cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fab400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac01c030 con 0x557c6b1b98d0 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb8004040 con 0x557c6b1b98d0 2024-04-20T21:42:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3fac004b20 con 0x557c6b1b98d0 2024-04-20T21:42:37.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fac026050 con 0x557c6b1b98d0 2024-04-20T21:42:37.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fcaf9b740 10 client.26990 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.193+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c6b0a9820 con 0x557c6b1b98d0 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8004040 con 0x557c6b1b98d0 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3fac00b040 con 0x557c6b1b98d0 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 10 client.26990 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 10 client.26990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 20 client.26990 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b1d2490 0x557c6b1d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 4 client.26990 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:37.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6b12ba70 con 0x557c6b1d2490 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.194+0000 7f3fc6200640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b1d2490 0x557c6b1d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.195+0000 7f3fc6200640 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b1d2490 0x557c6b1d48f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4b558372892ff29 server_cookie=87a0c66b7a04b43d in_seq=0 out_seq=0 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.195+0000 7f3fab400640 10 client.26990 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fab400640 1 -- 172.21.15.7:0/4031892871 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c6b12ba70 con 0x557c6b1d2490 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fab400640 10 client.26990 handle_command_reply: tid=0 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 client.26990 shutdown 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 msgr2=0x557c6b1bfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3fac005fd0 tx=0x7f3fac00d7b0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 shutdown_connections 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b1d2490 0x557c6b1d48f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1bfa40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fab400640 0 client.26990 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/4031892871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 >> 172.21.15.7:0/4031892871 conn(0x557c6b1b36e0 msgr2=0x557c6b1b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 shutdown_connections 2024-04-20T21:42:37.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/4031892871 wait complete. 2024-04-20T21:42:37.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.196+0000 7f3fcaf9b740 20 client.26990 trim_cache size 0 max 16384 2024-04-20T21:42:37.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 Processor -- start 2024-04-20T21:42:37.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 -- start start 2024-04-20T21:42:37.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b2790 con 0x557c6b1b8fb0 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01b0050 con 0x557c6b1b98d0 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.211+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01adec0 con 0x557c6b1c24b0 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45462/0 (socket says 172.21.15.7:45462) 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc6200640 1 -- 172.21.15.7:0/1676215526 learned_addr learned my addr 172.21.15.7:0/1676215526 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc6200640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 -- 172.21.15.7:0/1676215526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 msgr2=0x557c6b1bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 -- 172.21.15.7:0/1676215526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 msgr2=0x557c6b1c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 -- 172.21.15.7:0/1676215526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6ae303f0 con 0x557c6b1b98d0 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc6200640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.212+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1ce130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3fac021640 tx=0x7f3fac00dcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3c914485ded64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fb6000640 1 -- 172.21.15.7:0/1676215526 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac02e020 con 0x557c6b1b98d0 2024-04-20T21:42:37.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fb6000640 1 -- 172.21.15.7:0/1676215526 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fac021ab0 con 0x557c6b1b98d0 2024-04-20T21:42:37.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fb6000640 1 -- 172.21.15.7:0/1676215526 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac01c040 con 0x557c6b1b98d0 2024-04-20T21:42:37.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/1676215526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 msgr2=0x557c6b1ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1ce130 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3fac021640 tx=0x7f3fac00dcb0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/1676215526 shutdown_connections 2024-04-20T21:42:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1b98d0 0x557c6b1ce130 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1c24b0 0x557c6b1bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/1676215526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.213+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/1676215526 >> 172.21.15.7:0/1676215526 conn(0x557c6b1b36e0 msgr2=0x557c6b1bf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.214+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/1676215526 shutdown_connections 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.214+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/1676215526 wait complete. 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.214+0000 7f3fcaf9b740 1 -- start start 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.214+0000 7f3fcaf9b740 1 Processor -- start 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 0x557c6b1bc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01ae510 con 0x557c6b1b8fb0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01af980 con 0x557c6b1c24b0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc01aebe0 con 0x557c6b1b98d0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fcaf9b740 10 client.0 _ll_register_callbacks cb 0x7f3fac00e4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 0x557c6b1bc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45476/0 (socket says 172.21.15.7:45476) 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc6200640 1 -- 172.21.15.7:0/3798206894 learned_addr learned my addr 172.21.15.7:0/3798206894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 -- 172.21.15.7:0/3798206894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 msgr2=0x557c6b1bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 -- 172.21.15.7:0/3798206894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 msgr2=0x557c6b1bc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc4e00640 1 -- 172.21.15.7:0/3798206894 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6aefa7d0 con 0x557c6b1b98d0 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.215+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fc4e00640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 0x557c6b1bc600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x557c6b1af060 tx=0x7f3fac0093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53d0da4c1a479db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fab400640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac02e030 con 0x557c6b1b98d0 2024-04-20T21:42:37.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fac021ab0 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3fac018b20 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3fac042050 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fcaf9b740 10 client.26944 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c6b238a60 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.216+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fac021ab0 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3fac00b040 con 0x557c6b1b98d0 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 10 client.26944 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 10 client.26944 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 20 client.26944 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b239110 0x557c6b23b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 4 client.26944 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c6b1258e0 con 0x557c6b239110 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.217+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b239110 0x557c6b23b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fc6200640 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b239110 0x557c6b23b500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f53cabdcef7f9292 server_cookie=1e456c9746a34aff in_seq=0 out_seq=0 2024-04-20T21:42:37.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fab400640 10 client.26944 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fab400640 1 -- 172.21.15.7:0/3798206894 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c6b1258e0 con 0x557c6b239110 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fab400640 10 client.26944 handle_command_reply: tid=0 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 client.26944 shutdown 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 msgr2=0x557c6b1bc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 0x557c6b1bc600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x557c6b1af060 tx=0x7f3fac0093a0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 shutdown_connections 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x557c6b239110 0x557c6b23b500 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557c6b1c24b0 0x557c6b1bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557c6b1b98d0 0x557c6b1bc600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 --2- 172.21.15.7:0/3798206894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557c6b1b8fb0 0x557c6b1bc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 >> 172.21.15.7:0/3798206894 conn(0x557c6b1b36e0 msgr2=0x557c6b1bfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fab400640 0 client.26944 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 shutdown_connections 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.218+0000 7f3fcaf9b740 1 -- 172.21.15.7:0/3798206894 wait complete. 2024-04-20T21:42:37.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fcaf9b740 20 client.26944 trim_cache size 0 max 16384 2024-04-20T21:42:37.229 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 msgr2=0x7f3fa40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 0x7f3fa40a6260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x557c6b1b8a80 tx=0x7f3fb00d4570 comp rx=0 tx=0).stop 2024-04-20T21:42:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 msgr2=0x7f3fc01a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000e9f0 tx=0x7f3fb000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 shutdown_connections 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fc019a810 0x7f3fc019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fc0107260 0x7f3fc01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3fa40a3da0 0x7f3fa40a6260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 --2- 172.21.15.7:0/471803065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc0102190 0x7f3fc01a06a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 >> 172.21.15.7:0/471803065 conn(0x7f3fc00fda10 msgr2=0x7f3fc00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 shutdown_connections 2024-04-20T21:42:37.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.219+0000 7f3fc8200640 1 -- 172.21.15.7:0/471803065 wait complete. 2024-04-20T21:42:37.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 -- 172.21.15.7:0/3187104441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 msgr2=0x7f127c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 --2- 172.21.15.7:0/3187104441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c076870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1264009860 tx=0x7f126402bc00 comp rx=0 tx=0).stop 2024-04-20T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 -- 172.21.15.7:0/3187104441 shutdown_connections 2024-04-20T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 --2- 172.21.15.7:0/3187104441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c112bd0 0x7f127c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 --2- 172.21.15.7:0/3187104441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c076db0 0x7f127c112690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 --2- 172.21.15.7:0/3187104441 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c076870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 -- 172.21.15.7:0/3187104441 >> 172.21.15.7:0/3187104441 conn(0x7f127c0ffa40 msgr2=0x7f127c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 -- 172.21.15.7:0/3187104441 shutdown_connections 2024-04-20T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.424+0000 7f1282600640 1 -- 172.21.15.7:0/3187104441 wait complete. 2024-04-20T21:42:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.425+0000 7f1282600640 1 Processor -- start 2024-04-20T21:42:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.425+0000 7f1282600640 1 -- start start 2024-04-20T21:42:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 0x7f127c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f127c118700 con 0x7f127c112bd0 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f127c118580 con 0x7f127c076db0 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1282600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f127c118880 con 0x7f127c076490 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 0x7f127c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1273e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1273e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42856/0 (socket says 172.21.15.7:42856) 2024-04-20T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1273e00640 1 -- 172.21.15.7:0/1453731397 learned_addr learned my addr 172.21.15.7:0/1453731397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281200640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 -- 172.21.15.7:0/1453731397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 msgr2=0x7f127c1a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c1a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 -- 172.21.15.7:0/1453731397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 msgr2=0x7f127c1a4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281c00640 1 -- 172.21.15.7:0/1453731397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c19f240 con 0x7f127c112bd0 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1281200640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c1a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:37.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.426+0000 7f1273e00640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f1281c00640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 0x7f127c19eac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1274009fd0 tx=0x7f127400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f127a000640 1 -- 172.21.15.7:0/1453731397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127400ce20 con 0x7f127c112bd0 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c19f530 con 0x7f127c112bd0 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f127a000640 1 -- 172.21.15.7:0/1453731397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1274005020 con 0x7f127c112bd0 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1ab7d0 con 0x7f127c112bd0 2024-04-20T21:42:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.427+0000 7f127a000640 1 -- 172.21.15.7:0/1453731397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740106c0 con 0x7f127c112bd0 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.430+0000 7f1286993740 1 Processor -- start 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.430+0000 7f127a000640 1 -- 172.21.15.7:0/1453731397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1274005540 con 0x7f127c112bd0 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.430+0000 7f1286993740 1 -- start start 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a4ae71050 con 0x555a4af15670 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a4ae17490 con 0x555a4af0c170 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1286993740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a4ae6d010 con 0x555a4af0ca90 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1273e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1273e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45506/0 (socket says 172.21.15.7:45506) 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1273e00640 1 -- 172.21.15.7:0/1284266477 learned_addr learned my addr 172.21.15.7:0/1284266477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1281200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.431+0000 7f1273e00640 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 -- 172.21.15.7:0/1284266477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 msgr2=0x555a4af15130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af15130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 -- 172.21.15.7:0/1284266477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 msgr2=0x555a4af0c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af0c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 -- 172.21.15.7:0/1284266477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4ae1f070 con 0x555a4af15670 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1281200640 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1273e00640 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f126c011c50 tx=0x7f126c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7225f11e695c0ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f127a000640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 0x7f125c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1272000640 1 -- 172.21.15.7:0/1284266477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c00fc50 con 0x555a4af15670 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f127a000640 1 -- 172.21.15.7:0/1453731397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f12740d0e80 con 0x7f127c112bd0 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1272000640 1 -- 172.21.15.7:0/1284266477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f126c00fdf0 con 0x555a4af15670 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1272000640 1 -- 172.21.15.7:0/1284266477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c0049c0 con 0x555a4af15670 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.432+0000 7f1281200640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 0x7f125c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 -- 172.21.15.7:0/1284266477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 msgr2=0x555a4af17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f126c011c50 tx=0x7f126c009bd0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1281200640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 0x7f125c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1264009830 tx=0x7f1264000bc0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 -- 172.21.15.7:0/1284266477 shutdown_connections 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af17a60 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 --2- 172.21.15.7:0/1284266477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af0c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 -- 172.21.15.7:0/1284266477 >> 172.21.15.7:0/1284266477 conn(0x555a4af068a0 msgr2=0x555a4af06c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 -- 172.21.15.7:0/1284266477 shutdown_connections 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.433+0000 7f1286993740 1 -- 172.21.15.7:0/1284266477 wait complete. 2024-04-20T21:42:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 -- start start 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 Processor -- start 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0ca90 0x555a4af12c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af15670 0x555a4af131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x555a4af18460 con 0x555a4af0c170 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555a4ae2a860 con 0x555a4af0ca90 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x555a4af185e0 con 0x555a4af15670 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.434+0000 7f1286993740 10 client.0 _ll_register_callbacks cb 0x555a4adf8fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45512/0 (socket says 172.21.15.7:45512) 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 -- 172.21.15.7:0/2823932496 learned_addr learned my addr 172.21.15.7:0/2823932496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1273e00640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0ca90 0x555a4af12c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281c00640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af15670 0x555a4af131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 -- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af15670 msgr2=0x555a4af131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af15670 0x555a4af131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 -- 172.21.15.7:0/2823932496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0ca90 msgr2=0x555a4af12c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0ca90 0x555a4af12c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 -- 172.21.15.7:0/2823932496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4ae2e950 con 0x555a4af0c170 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f1281200640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1264009860 tx=0x7f126402bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887ffc5373200232 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.435+0000 7f126aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264039040 con 0x555a4af0c170 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f1264005240 con 0x555a4af0c170 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f126c00fdf0 con 0x555a4af0c170 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1264058050 con 0x555a4af0c170 2024-04-20T21:42:37.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f1286993740 10 client.16797 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4adfc820 con 0x555a4af0c170 2024-04-20T21:42:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.436+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f126c00fdf0 con 0x555a4af0c170 2024-04-20T21:42:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f12640047e0 con 0x555a4af0c170 2024-04-20T21:42:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 10 client.16797 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 20 client.16797 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af25790 0x555a4af27c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 4 client.16797 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a4ae7ea70 con 0x555a4af25790 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.437+0000 7f1273e00640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af25790 0x555a4af27c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.438+0000 7f1273e00640 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af25790 0x555a4af27c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ad22268605987384 server_cookie=82318191f79f7b4b in_seq=0 out_seq=0 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.438+0000 7f126aa00640 10 client.16797 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f126aa00640 1 -- 172.21.15.7:0/2823932496 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a4ae7ea70 con 0x555a4af25790 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f126aa00640 10 client.16797 handle_command_reply: tid=0 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 client.16797 shutdown 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 msgr2=0x555a4af12730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1264009860 tx=0x7f126402bc00 comp rx=0 tx=0).stop 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 shutdown_connections 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af25790 0x555a4af27c30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0ca90 0x555a4af12c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af15670 0x555a4af131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f126aa00640 0 client.16797 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 --2- 172.21.15.7:0/2823932496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af12730 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 >> 172.21.15.7:0/2823932496 conn(0x555a4af068a0 msgr2=0x555a4af085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 shutdown_connections 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 1 -- 172.21.15.7:0/2823932496 wait complete. 2024-04-20T21:42:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.439+0000 7f1286993740 20 client.16797 trim_cache size 0 max 16384 2024-04-20T21:42:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.453+0000 7f1286993740 1 Processor -- start 2024-04-20T21:42:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.453+0000 7f1286993740 1 -- start start 2024-04-20T21:42:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.453+0000 7f1286993740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af21320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1286993740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1286993740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af15670 0x555a4af2d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1286993740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b68f0 con 0x555a4af0c170 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1286993740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b41b0 con 0x555a4af15670 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1286993740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b2020 con 0x555a4af0ca90 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58270/0 (socket says 172.21.15.7:58270) 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 -- 172.21.15.7:0/4264674333 learned_addr learned my addr 172.21.15.7:0/4264674333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1281c00640 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af15670 0x555a4af2d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1281200640 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af21320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 -- 172.21.15.7:0/4264674333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af15670 msgr2=0x555a4af2d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af15670 0x555a4af2d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 -- 172.21.15.7:0/4264674333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 msgr2=0x555a4af21320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af21320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.454+0000 7f1273e00640 1 -- 172.21.15.7:0/4264674333 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a4ab833f0 con 0x555a4af0ca90 2024-04-20T21:42:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.455+0000 7f1273e00640 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f126c005520 tx=0x7f126c0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=637d56e5e5b43d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.455+0000 7f1271600640 1 -- 172.21.15.7:0/4264674333 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c02d170 con 0x555a4af0ca90 2024-04-20T21:42:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.455+0000 7f1271600640 1 -- 172.21.15.7:0/4264674333 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f126c02d7f0 con 0x555a4af0ca90 2024-04-20T21:42:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.455+0000 7f1271600640 1 -- 172.21.15.7:0/4264674333 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c01f9e0 con 0x555a4af0ca90 2024-04-20T21:42:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- 172.21.15.7:0/4264674333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 msgr2=0x555a4af057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f126c005520 tx=0x7f126c0055f0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- 172.21.15.7:0/4264674333 shutdown_connections 2024-04-20T21:42:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af15670 0x555a4af2d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af057e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 --2- 172.21.15.7:0/4264674333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af0c170 0x555a4af21320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- 172.21.15.7:0/4264674333 >> 172.21.15.7:0/4264674333 conn(0x555a4af068a0 msgr2=0x555a4af050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- 172.21.15.7:0/4264674333 shutdown_connections 2024-04-20T21:42:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- 172.21.15.7:0/4264674333 wait complete. 2024-04-20T21:42:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 -- start start 2024-04-20T21:42:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.456+0000 7f1286993740 1 Processor -- start 2024-04-20T21:42:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af1d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af1a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b2670 con 0x555a4af15670 2024-04-20T21:42:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b3ae0 con 0x555a4af0c170 2024-04-20T21:42:37.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f127c1b2d40 con 0x555a4af0ca90 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1286993740 10 client.0 _ll_register_callbacks cb 0x7f12640096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1281c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af1a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58280/0 (socket says 172.21.15.7:58280) 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 -- 172.21.15.7:0/2430296251 learned_addr learned my addr 172.21.15.7:0/2430296251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1281200640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af1d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 -- 172.21.15.7:0/2430296251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 msgr2=0x555a4af1d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af1d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 -- 172.21.15.7:0/2430296251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 msgr2=0x555a4af1a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af1a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1273e00640 1 -- 172.21.15.7:0/2430296251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a4ac4d7d0 con 0x555a4af0ca90 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.457+0000 7f1281c00640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af1a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f1273e00640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f126c009c10 tx=0x7f126c005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f526341769bf9af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f126aa00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126c02d170 con 0x555a4af0ca90 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f126c02d7f0 con 0x555a4af0ca90 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f126c02d310 con 0x555a4af0ca90 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f126c02d510 con 0x555a4af0ca90 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f1286993740 10 client.26962 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.458+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a4af8c4e0 con 0x555a4af0ca90 2024-04-20T21:42:37.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f126c02d7f0 con 0x555a4af0ca90 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f126c03e020 con 0x555a4af0ca90 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 10 client.26962 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 10 client.26962 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 20 client.26962 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af8c9c0 0x555a4af8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 4 client.26962 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a4ae788e0 con 0x555a4af8c9c0 2024-04-20T21:42:37.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.459+0000 7f1281200640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af8c9c0 0x555a4af8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.460+0000 7f1281200640 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af8c9c0 0x555a4af8ede0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff58aa76aee00375 server_cookie=3345bd7f8221bd3f in_seq=0 out_seq=0 2024-04-20T21:42:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.460+0000 7f126aa00640 10 client.26962 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.460+0000 7f126aa00640 1 -- 172.21.15.7:0/2430296251 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a4ae788e0 con 0x555a4af8c9c0 2024-04-20T21:42:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.460+0000 7f126aa00640 10 client.26962 handle_command_reply: tid=0 2024-04-20T21:42:37.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.460+0000 7f1286993740 1 client.26962 shutdown 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 msgr2=0x555a4af1dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f126c009c10 tx=0x7f126c005e60 comp rx=0 tx=0).stop 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 shutdown_connections 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x555a4af8c9c0 0x555a4af8ede0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x555a4af15670 0x555a4af1a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555a4af0ca90 0x555a4af1dd70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f126aa00640 0 client.26962 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 --2- 172.21.15.7:0/2430296251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555a4af0c170 0x555a4af1d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 >> 172.21.15.7:0/2430296251 conn(0x555a4af068a0 msgr2=0x555a4af126d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 shutdown_connections 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 1 -- 172.21.15.7:0/2430296251 wait complete. 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1286993740 20 client.26962 trim_cache size 0 max 16384 2024-04-20T21:42:37.472 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 msgr2=0x7f125c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.461+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 0x7f125c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1264009830 tx=0x7f1264000bc0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 msgr2=0x7f127c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 0x7f127c19eac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1274009fd0 tx=0x7f127400ef10 comp rx=0 tx=0).stop 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 shutdown_connections 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f125c0a3d20 0x7f125c0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f127c076db0 0x7f127c1a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f127c112bd0 0x7f127c19eac0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 --2- 172.21.15.7:0/1453731397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f127c076490 0x7f127c1a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 >> 172.21.15.7:0/1453731397 conn(0x7f127c0ffa40 msgr2=0x7f127c114600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 shutdown_connections 2024-04-20T21:42:37.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.462+0000 7f1282600640 1 -- 172.21.15.7:0/1453731397 wait complete. 2024-04-20T21:42:37.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:42:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 -- 172.21.15.7:0/3285991706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946810e3e0 msgr2=0x7f946810e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 --2- 172.21.15.7:0/3285991706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946810e3e0 0x7f946810e7c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9458009890 tx=0x7f945802bb70 comp rx=0 tx=0).stop 2024-04-20T21:42:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 -- 172.21.15.7:0/3285991706 shutdown_connections 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 --2- 172.21.15.7:0/3285991706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94681084a0 0x7f9468108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 --2- 172.21.15.7:0/3285991706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f9468115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 --2- 172.21.15.7:0/3285991706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946810e3e0 0x7f946810e7c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 -- 172.21.15.7:0/3285991706 >> 172.21.15.7:0/3285991706 conn(0x7f9468077d70 msgr2=0x7f9468106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 -- 172.21.15.7:0/3285991706 shutdown_connections 2024-04-20T21:42:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.680+0000 7f9467e00640 1 -- 172.21.15.7:0/3285991706 wait complete. 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.681+0000 7f9467e00640 1 Processor -- start 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.681+0000 7f9467e00640 1 -- start start 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f946819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 0x7f94681a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9468118710 con 0x7f94681084a0 2024-04-20T21:42:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9468118590 con 0x7f946810e3e0 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f9468118890 con 0x7f9468108e40 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45522/0 (socket says 172.21.15.7:45522) 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 -- 172.21.15.7:0/1497857655 learned_addr learned my addr 172.21.15.7:0/1497857655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466000640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f946819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9467400640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 0x7f94681a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 -- 172.21.15.7:0/1497857655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 msgr2=0x7f946819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f946819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 -- 172.21.15.7:0/1497857655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 msgr2=0x7f94681a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 0x7f94681a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.682+0000 7f9466a00640 1 -- 172.21.15.7:0/1497857655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94681a3d40 con 0x7f94681084a0 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f9466000640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f946819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f9467400640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 0x7f94681a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f9466a00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f9458004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f945f400640 1 -- 172.21.15.7:0/1497857655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458041020 con 0x7f94681084a0 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f945f400640 1 -- 172.21.15.7:0/1497857655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945802c9f0 con 0x7f94681084a0 2024-04-20T21:42:37.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f945f400640 1 -- 172.21.15.7:0/1497857655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458039040 con 0x7f94681084a0 2024-04-20T21:42:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94681a3fd0 con 0x7f94681084a0 2024-04-20T21:42:37.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.683+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681ab8c0 con 0x7f94681084a0 2024-04-20T21:42:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.687+0000 7f9470604740 1 Processor -- start 2024-04-20T21:42:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 -- start start 2024-04-20T21:42:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b267550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 0x55ae6b270130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 0x55ae6b272a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b1cc050 con 0x55ae6b267a90 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b172490 con 0x55ae6b267170 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f9470604740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b1c8010 con 0x55ae6b270670 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.688+0000 7f945f400640 1 -- 172.21.15.7:0/1497857655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f9458044020 con 0x7f94681084a0 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 0x55ae6b270130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9466000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 0x55ae6b272a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b267550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9466a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 0x55ae6b270130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45536/0 (socket says 172.21.15.7:45536) 2024-04-20T21:42:37.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9466a00640 1 -- 172.21.15.7:0/1151289889 learned_addr learned my addr 172.21.15.7:0/1151289889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 -- 172.21.15.7:0/1151289889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 msgr2=0x55ae6b272a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 0x55ae6b272a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 -- 172.21.15.7:0/1151289889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 msgr2=0x55ae6b270130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 0x55ae6b270130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 -- 172.21.15.7:0/1151289889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6b17a070 con 0x55ae6b267170 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9466000640 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 0x55ae6b272a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.689+0000 7f9467400640 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b267550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f9460011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8165359431e0074 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f9453e00640 1 -- 172.21.15.7:0/1151289889 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460004d20 con 0x55ae6b267170 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f9453e00640 1 -- 172.21.15.7:0/1151289889 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946000f9a0 con 0x55ae6b267170 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f9453e00640 1 -- 172.21.15.7:0/1151289889 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94600045b0 con 0x55ae6b267170 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f945f400640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 0x7f944c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f945f400640 1 -- 172.21.15.7:0/1497857655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f94580f1ed0 con 0x7f94681084a0 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.690+0000 7f9466a00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 0x7f944c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- 172.21.15.7:0/1151289889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 msgr2=0x55ae6b267550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b267550 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f9460011bf0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- 172.21.15.7:0/1151289889 shutdown_connections 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b270670 0x55ae6b272a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267a90 0x55ae6b270130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 --2- 172.21.15.7:0/1151289889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b267550 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- 172.21.15.7:0/1151289889 >> 172.21.15.7:0/1151289889 conn(0x55ae6b2618a0 msgr2=0x55ae6b261c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- 172.21.15.7:0/1151289889 shutdown_connections 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- 172.21.15.7:0/1151289889 wait complete. 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9466a00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 0x7f944c0a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f945802cbc0 tx=0x7f94580f5980 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.691+0000 7f9470604740 1 -- start start 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 Processor -- start 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b26dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b26e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b273460 con 0x55ae6b267170 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b185860 con 0x55ae6b270670 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ae6b2735e0 con 0x55ae6b267a90 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9470604740 10 client.0 _ll_register_callbacks cb 0x55ae6b153fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9467400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9467400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45546/0 (socket says 172.21.15.7:45546) 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.692+0000 7f9467400640 1 -- 172.21.15.7:0/4212666757 learned_addr learned my addr 172.21.15.7:0/4212666757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9466000640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b26dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b26e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 -- 172.21.15.7:0/4212666757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 msgr2=0x55ae6b26dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b26dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 -- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 msgr2=0x55ae6b26e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b26e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 -- 172.21.15.7:0/4212666757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6b189950 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9467400640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f94600041d0 tx=0x7f9460004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc72c6afb4dd53f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9450c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946001e040 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f9460011240 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.693+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f946000f9a0 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9460004a40 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 10 client.16815 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae6b157820 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946000f9a0 con 0x55ae6b267170 2024-04-20T21:42:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9460018070 con 0x55ae6b267170 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 10 client.16815 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 20 client.16815 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2805e0 0x55ae6b2829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 4 client.16815 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.694+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae6b1d9a70 con 0x55ae6b2805e0 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.695+0000 7f9466000640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2805e0 0x55ae6b2829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.695+0000 7f9466000640 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2805e0 0x55ae6b2829d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f686a070271e522 server_cookie=4db4d2467de0322f in_seq=0 out_seq=0 2024-04-20T21:42:37.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.695+0000 7f9450c00640 10 client.16815 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:37.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9450c00640 1 -- 172.21.15.7:0/4212666757 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae6b1d9a70 con 0x55ae6b2805e0 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9450c00640 10 client.16815 handle_command_reply: tid=0 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 client.16815 shutdown 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 msgr2=0x55ae6b26d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f94600041d0 tx=0x7f9460004230 comp rx=0 tx=0).stop 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 shutdown_connections 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b26e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2805e0 0x55ae6b2829d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b26dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 --2- 172.21.15.7:0/4212666757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26d700 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 >> 172.21.15.7:0/4212666757 conn(0x55ae6b2618a0 msgr2=0x55ae6b2635f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.696+0000 7f9450c00640 0 client.16815 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.697+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 shutdown_connections 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.697+0000 7f9470604740 1 -- 172.21.15.7:0/4212666757 wait complete. 2024-04-20T21:42:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.697+0000 7f9470604740 20 client.16815 trim_cache size 0 max 16384 2024-04-20T21:42:37.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 Processor -- start 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 -- start start 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b27c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b263350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b6960 con 0x55ae6b267170 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b4220 con 0x55ae6b270670 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9470604740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b2090 con 0x55ae6b267a90 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9467400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9466000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b27c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9467400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45554/0 (socket says 172.21.15.7:45554) 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.712+0000 7f9467400640 1 -- 172.21.15.7:0/286915832 learned_addr learned my addr 172.21.15.7:0/286915832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9467400640 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b263350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 -- 172.21.15.7:0/286915832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 msgr2=0x55ae6b263350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b263350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 -- 172.21.15.7:0/286915832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 msgr2=0x55ae6b26fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 -- 172.21.15.7:0/286915832 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6aede3f0 con 0x55ae6b267a90 2024-04-20T21:42:37.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9467400640 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:37.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9466000640 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b27c350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55ae6b26cf60 tx=0x7f9454009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8efbe191b93f06c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.713+0000 7f9453400640 1 -- 172.21.15.7:0/286915832 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454023020 con 0x55ae6b267a90 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9453400640 1 -- 172.21.15.7:0/286915832 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945401f0d0 con 0x55ae6b267a90 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9453400640 1 -- 172.21.15.7:0/286915832 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945401d680 con 0x55ae6b267a90 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 -- 172.21.15.7:0/286915832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 msgr2=0x55ae6b27c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b27c350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55ae6b26cf60 tx=0x7f9454009260 comp rx=0 tx=0).stop 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 -- 172.21.15.7:0/286915832 shutdown_connections 2024-04-20T21:42:37.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b270670 0x55ae6b263350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b27c350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 --2- 172.21.15.7:0/286915832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b267170 0x55ae6b26fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 -- 172.21.15.7:0/286915832 >> 172.21.15.7:0/286915832 conn(0x55ae6b2618a0 msgr2=0x55ae6b262c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 -- 172.21.15.7:0/286915832 shutdown_connections 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.714+0000 7f9470604740 1 -- 172.21.15.7:0/286915832 wait complete. 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 -- start start 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 Processor -- start 2024-04-20T21:42:37.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b273f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b270670 0x55ae6b274440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b26e0 con 0x55ae6b270670 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b3b50 con 0x55ae6b267170 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b2db0 con 0x55ae6b267a90 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.715+0000 7f9470604740 10 client.0 _ll_register_callbacks cb 0x7f9460011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b270670 0x55ae6b274440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42980/0 (socket says 172.21.15.7:42980) 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 -- 172.21.15.7:0/1891435151 learned_addr learned my addr 172.21.15.7:0/1891435151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9466000640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b273f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 -- 172.21.15.7:0/1891435151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 msgr2=0x55ae6b273f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b273f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 -- 172.21.15.7:0/1891435151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b270670 msgr2=0x55ae6b274440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b270670 0x55ae6b274440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 -- 172.21.15.7:0/1891435151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6afa87d0 con 0x55ae6b267170 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9467400640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9460003a00 tx=0x7f94600021f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf9720105b33c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:37.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.716+0000 7f9450c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460037040 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f9460004be0 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945401f0d0 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f946001e040 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9470604740 10 client.27032 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ae6b2e6780 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f945401f0d0 con 0x55ae6b267170 2024-04-20T21:42:37.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.717+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9460018040 con 0x55ae6b267170 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 10 client.27032 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 10 client.27032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 20 client.27032 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2e6e70 0x55ae6b2e9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 4 client.27032 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:42:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae6b269300 con 0x55ae6b2e6e70 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9466000640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2e6e70 0x55ae6b2e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9466000640 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2e6e70 0x55ae6b2e9290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32322430ab03fe4e server_cookie=af47fdca01b07b60 in_seq=0 out_seq=0 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.718+0000 7f9450c00640 10 client.27032 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9450c00640 1 -- 172.21.15.7:0/1891435151 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae6b269300 con 0x55ae6b2e6e70 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9450c00640 10 client.27032 handle_command_reply: tid=0 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 client.27032 shutdown 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 msgr2=0x55ae6b278b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9460003a00 tx=0x7f94600021f0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 shutdown_connections 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ae6b2e6e70 0x55ae6b2e9290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ae6b270670 0x55ae6b274440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6b267a90 0x55ae6b273f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 --2- 172.21.15.7:0/1891435151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ae6b267170 0x55ae6b278b40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9450c00640 0 client.27032 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 >> 172.21.15.7:0/1891435151 conn(0x55ae6b2618a0 msgr2=0x55ae6b2634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 shutdown_connections 2024-04-20T21:42:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 1 -- 172.21.15.7:0/1891435151 wait complete. 2024-04-20T21:42:37.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.719+0000 7f9470604740 20 client.27032 trim_cache size 0 max 16384 2024-04-20T21:42:37.726 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:42:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 msgr2=0x7f944c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 0x7f944c0a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f945802cbc0 tx=0x7f94580f5980 comp rx=0 tx=0).stop 2024-04-20T21:42:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 msgr2=0x7f946819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f9458004550 comp rx=0 tx=0).stop 2024-04-20T21:42:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 shutdown_connections 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f944c0a3da0 0x7f944c0a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946810e3e0 0x7f94681a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9468108e40 0x7f946819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 --2- 172.21.15.7:0/1497857655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94681084a0 0x7f946819ed50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 >> 172.21.15.7:0/1497857655 conn(0x7f9468077d70 msgr2=0x7f9468078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 shutdown_connections 2024-04-20T21:42:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:37.720+0000 7f9467e00640 1 -- 172.21.15.7:0/1497857655 wait complete. 2024-04-20T21:42:37.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1605791986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:37.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1605791986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:38.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1605791986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:42:38.738 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:42:38.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:42:38.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:42:38.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:38 smithi016 ceph-mon[21409]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:38.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:38 smithi186 ceph-mon[22157]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 -- 172.21.15.7:0/1444255334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074300 msgr2=0x7f3d400746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1444255334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074300 0x7f3d400746e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009860 tx=0x7f3d3002bc10 comp rx=0 tx=0).stop 2024-04-20T21:42:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 -- 172.21.15.7:0/1444255334 shutdown_connections 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1444255334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40112d40 0x7f3d40115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1444255334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40074c20 0x7f3d40112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1444255334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074300 0x7f3d400746e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.970+0000 7f3d44c00640 1 -- 172.21.15.7:0/1444255334 >> 172.21.15.7:0/1444255334 conn(0x7f3d400fd840 msgr2=0x7f3d400ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.971+0000 7f3d44c00640 1 -- 172.21.15.7:0/1444255334 shutdown_connections 2024-04-20T21:42:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.971+0000 7f3d44c00640 1 -- 172.21.15.7:0/1444255334 wait complete. 2024-04-20T21:42:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.972+0000 7f3d44c00640 1 Processor -- start 2024-04-20T21:42:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.972+0000 7f3d44c00640 1 -- start start 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 0x7f3d401a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40112d40 0x7f3d4019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40118710 con 0x7f3d40112d40 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40118590 con 0x7f3d40074300 2024-04-20T21:42:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d44c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40118890 con 0x7f3d40074c20 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40112d40 0x7f3d4019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 0x7f3d401a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:42992/0 (socket says 172.21.15.7:42992) 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 -- 172.21.15.7:0/1529329119 learned_addr learned my addr 172.21.15.7:0/1529329119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 -- 172.21.15.7:0/1529329119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 msgr2=0x7f3d401a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 0x7f3d401a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 -- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40112d40 msgr2=0x7f3d4019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40112d40 0x7f3d4019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3f400640 1 -- 172.21.15.7:0/1529329119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4019f230 con 0x7f3d40074300 2024-04-20T21:42:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.973+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 0x7f3d401a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d3f400640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d30037040 tx=0x7f3d300346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d37e00640 1 -- 172.21.15.7:0/1529329119 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30041020 con 0x7f3d40074300 2024-04-20T21:42:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4019f4c0 con 0x7f3d40074300 2024-04-20T21:42:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d37e00640 1 -- 172.21.15.7:0/1529329119 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d300044d0 con 0x7f3d40074300 2024-04-20T21:42:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d37e00640 1 -- 172.21.15.7:0/1529329119 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30039040 con 0x7f3d40074300 2024-04-20T21:42:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.974+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d401ab7c0 con 0x7f3d40074300 2024-04-20T21:42:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 Processor -- start 2024-04-20T21:42:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 -- start start 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff3a10 0x56486effc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56486ef57d20 con 0x56486eff30f0 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56486eefe3e0 con 0x56486effc5f0 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d48e55740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56486ef53ce0 con 0x56486eff3a10 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d37e00640 1 -- 172.21.15.7:0/1529329119 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f3d30044020 con 0x7f3d40074300 2024-04-20T21:42:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45584/0 (socket says 172.21.15.7:45584) 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.978+0000 7f3d3fe00640 1 -- 172.21.15.7:0/3025782858 learned_addr learned my addr 172.21.15.7:0/3025782858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3f400640 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff3a10 0x56486effc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 -- 172.21.15.7:0/3025782858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff3a10 msgr2=0x56486effc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff3a10 0x56486effc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 -- 172.21.15.7:0/3025782858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 msgr2=0x56486effe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 -- 172.21.15.7:0/3025782858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486ef1d980 con 0x56486eff30f0 2024-04-20T21:42:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.979+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009f60 tx=0x7f3d38009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8ca13dad2d2a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d2be00640 1 -- 172.21.15.7:0/3025782858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003b70 con 0x56486eff30f0 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d2be00640 1 -- 172.21.15.7:0/3025782858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d38015070 con 0x56486eff30f0 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d2be00640 1 -- 172.21.15.7:0/3025782858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38011880 con 0x56486eff30f0 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d37e00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 0x7f3d240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d48e55740 1 -- 172.21.15.7:0/3025782858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 msgr2=0x56486eff34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d48e55740 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009f60 tx=0x7f3d38009f90 comp rx=0 tx=0).stop 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d37e00640 1 -- 172.21.15.7:0/1529329119 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3d300f1f60 con 0x7f3d40074300 2024-04-20T21:42:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.980+0000 7f3d3f400640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 0x7f3d240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 -- 172.21.15.7:0/3025782858 shutdown_connections 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff3a10 0x56486effc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 --2- 172.21.15.7:0/3025782858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff34d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 -- 172.21.15.7:0/3025782858 >> 172.21.15.7:0/3025782858 conn(0x56486efed820 msgr2=0x56486efedc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 -- 172.21.15.7:0/3025782858 shutdown_connections 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 -- 172.21.15.7:0/3025782858 wait complete. 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d3f400640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 0x7f3d240a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56486eff2bc0 tx=0x7f3d30005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 -- start start 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.981+0000 7f3d48e55740 1 Processor -- start 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 0x56486eff9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56486efff3e0 con 0x56486eff3a10 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56486ef117a0 con 0x56486effc5f0 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x56486efff560 con 0x56486eff30f0 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d48e55740 10 client.0 _ll_register_callbacks cb 0x56486ef13b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 0x56486eff9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 0x56486eff9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58358/0 (socket says 172.21.15.7:58358) 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d3fe00640 1 -- 172.21.15.7:0/2671543975 learned_addr learned my addr 172.21.15.7:0/2671543975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.982+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 -- 172.21.15.7:0/2671543975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 msgr2=0x56486eff9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 0x56486eff9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 -- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 msgr2=0x56486effa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 -- 172.21.15.7:0/2671543975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486ed327e0 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9bb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d40100010 tx=0x7f3d2c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618f58a68dce7ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d28c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c01e020 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3d2c0047a0 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.983+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d38015070 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d2c0049a0 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56486ed92f70 con 0x56486eff3a10 2024-04-20T21:42:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d38015070 con 0x56486eff3a10 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3d2c019070 con 0x56486eff3a10 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 20 client.16833 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f00caa0 0x56486f00eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 4 client.16833 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.984+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56486ef65740 con 0x56486f00caa0 2024-04-20T21:42:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.985+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f00caa0 0x56486f00eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.985+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f00caa0 0x56486f00eee0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925c7f5c23eb4644 server_cookie=62cd784933f3cdb1 in_seq=0 out_seq=0 2024-04-20T21:42:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.985+0000 7f3d28c00640 10 client.16833 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d28c00640 1 -- 172.21.15.7:0/2671543975 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56486ef65740 con 0x56486f00caa0 2024-04-20T21:42:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d28c00640 10 client.16833 handle_command_reply: tid=0 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 client.16833 shutdown 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 msgr2=0x56486eff9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9bb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3d40100010 tx=0x7f3d2c00f790 comp rx=0 tx=0).stop 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 shutdown_connections 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486effc5f0 0x56486effa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f00caa0 0x56486f00eee0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9bb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2671543975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486eff30f0 0x56486eff9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d28c00640 0 client.16833 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 >> 172.21.15.7:0/2671543975 conn(0x56486efed820 msgr2=0x56486efef570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.986+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 shutdown_connections 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.987+0000 7f3d48e55740 1 -- 172.21.15.7:0/2671543975 wait complete. 2024-04-20T21:42:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:38.987+0000 7f3d48e55740 20 client.16833 trim_cache size 0 max 16384 2024-04-20T21:42:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.006+0000 7f3d48e55740 1 Processor -- start 2024-04-20T21:42:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.006+0000 7f3d48e55740 1 -- start start 2024-04-20T21:42:39.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 0x56486f0082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f0147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b6980 con 0x56486eff3a10 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b4240 con 0x56486eff30f0 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d48e55740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b20b0 con 0x56486effc5f0 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 0x56486f0082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45604/0 (socket says 172.21.15.7:45604) 2024-04-20T21:42:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3ea00640 1 -- 172.21.15.7:0/2265179334 learned_addr learned my addr 172.21.15.7:0/2265179334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f0147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 -- 172.21.15.7:0/2265179334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 msgr2=0x56486f0147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f0147b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 -- 172.21.15.7:0/2265179334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 msgr2=0x56486eff9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.007+0000 7f3d3fe00640 1 -- 172.21.15.7:0/2265179334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56486eff3840 con 0x56486eff30f0 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.008+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 0x56486f0082d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x56486f00f750 tx=0x7f3d38000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95dae51ad4857010 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.008+0000 7f3d2b400640 1 -- 172.21.15.7:0/2265179334 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003970 con 0x56486eff30f0 2024-04-20T21:42:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.008+0000 7f3d2b400640 1 -- 172.21.15.7:0/2265179334 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d38015040 con 0x56486eff30f0 2024-04-20T21:42:39.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.008+0000 7f3d2b400640 1 -- 172.21.15.7:0/2265179334 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3802e040 con 0x56486eff30f0 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- 172.21.15.7:0/2265179334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 msgr2=0x56486f0082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 0x56486f0082d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x56486f00f750 tx=0x7f3d38000f40 comp rx=0 tx=0).stop 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- 172.21.15.7:0/2265179334 shutdown_connections 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f0147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff3a10 0x56486eff9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 --2- 172.21.15.7:0/2265179334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff30f0 0x56486f0082d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- 172.21.15.7:0/2265179334 >> 172.21.15.7:0/2265179334 conn(0x56486efed820 msgr2=0x56486efeca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- 172.21.15.7:0/2265179334 shutdown_connections 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- 172.21.15.7:0/2265179334 wait complete. 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 -- start start 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.009+0000 7f3d48e55740 1 Processor -- start 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 0x56486eff6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b2700 con 0x56486eff30f0 2024-04-20T21:42:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b3b70 con 0x56486eff3a10 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401b2dd0 con 0x56486effc5f0 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d48e55740 10 client.0 _ll_register_callbacks cb 0x7f3d2c011920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45608/0 (socket says 172.21.15.7:45608) 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d3fe00640 1 -- 172.21.15.7:0/1018292712 learned_addr learned my addr 172.21.15.7:0/1018292712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.010+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 0x56486eff6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 -- 172.21.15.7:0/1018292712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 msgr2=0x56486f072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 -- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 msgr2=0x56486eff6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 0x56486eff6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 -- 172.21.15.7:0/1018292712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56486ec2a170 con 0x56486eff30f0 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 0x56486eff6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d3fe00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3d380046b0 tx=0x7f3d38005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa5e155e8437d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d28c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3803f060 con 0x56486eff30f0 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f3d38011700 con 0x56486eff30f0 2024-04-20T21:42:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.011+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d38015040 con 0x56486eff30f0 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d38011900 con 0x56486eff30f0 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 10 client.16839 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56486f073630 con 0x56486eff30f0 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d38015040 con 0x56486eff30f0 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f3d38037070 con 0x56486eff30f0 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 10 client.16839 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 20 client.16839 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f073a90 0x56486f075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 4 client.16839 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:42:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.012+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56486ef5f5b0 con 0x56486f073a90 2024-04-20T21:42:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.013+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f073a90 0x56486f075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.013+0000 7f3d3ea00640 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f073a90 0x56486f075eb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca7be4ee6df998a1 server_cookie=a59e1361439ef41c in_seq=0 out_seq=0 2024-04-20T21:42:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.013+0000 7f3d28c00640 10 client.16839 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d28c00640 1 -- 172.21.15.7:0/1018292712 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56486ef5f5b0 con 0x56486f073a90 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d28c00640 10 client.16839 handle_command_reply: tid=0 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 client.16839 shutdown 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 msgr2=0x56486eff6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3d380046b0 tx=0x7f3d38005a70 comp rx=0 tx=0).stop 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 shutdown_connections 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56486eff3a10 0x56486eff6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x56486f073a90 0x56486f075eb0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56486effc5f0 0x56486f072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 --2- 172.21.15.7:0/1018292712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56486eff30f0 0x56486eff6a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 >> 172.21.15.7:0/1018292712 conn(0x56486efed820 msgr2=0x56486efeeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d28c00640 0 client.16839 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 shutdown_connections 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 1 -- 172.21.15.7:0/1018292712 wait complete. 2024-04-20T21:42:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.015+0000 7f3d48e55740 20 client.16839 trim_cache size 0 max 16384 2024-04-20T21:42:39.021 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:42:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "cc61036b-02ff-4119-8372-c1473e81cb55", 2024-04-20T21:42:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:42:39.021 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 msgr2=0x7f3d240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 0x7f3d240a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x56486eff2bc0 tx=0x7f3d30005f90 comp rx=0 tx=0).stop 2024-04-20T21:42:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 msgr2=0x7f3d401a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d30037040 tx=0x7f3d300346f0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 shutdown_connections 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d40112d40 0x7f3d4019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074c20 0x7f3d401a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f3d240a3d20 0x7f3d240a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 --2- 172.21.15.7:0/1529329119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d40074300 0x7f3d401a4900 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 >> 172.21.15.7:0/1529329119 conn(0x7f3d400fd840 msgr2=0x7f3d400ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 shutdown_connections 2024-04-20T21:42:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.016+0000 7f3d44c00640 1 -- 172.21.15.7:0/1529329119 wait complete. 2024-04-20T21:42:39.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:38 smithi007 ceph-mon[16867]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:39.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:42:39.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:42:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.242+0000 7fdf70600640 1 -- 172.21.15.7:0/2976310674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 msgr2=0x7fdf681061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.242+0000 7fdf70600640 1 --2- 172.21.15.7:0/2976310674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681061d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009860 tx=0x7fdf5402bb70 comp rx=0 tx=0).stop 2024-04-20T21:42:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 -- 172.21.15.7:0/2976310674 shutdown_connections 2024-04-20T21:42:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 --2- 172.21.15.7:0/2976310674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681061d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 --2- 172.21.15.7:0/2976310674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68106710 0x7fdf68111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 --2- 172.21.15.7:0/2976310674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68104490 0x7fdf68104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 -- 172.21.15.7:0/2976310674 >> 172.21.15.7:0/2976310674 conn(0x7fdf680ffbf0 msgr2=0x7fdf68102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 -- 172.21.15.7:0/2976310674 shutdown_connections 2024-04-20T21:42:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.243+0000 7fdf70600640 1 -- 172.21.15.7:0/2976310674 wait complete. 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.244+0000 7fdf70600640 1 Processor -- start 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.244+0000 7fdf70600640 1 -- start start 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 0x7fdf6819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681145e0 con 0x7fdf68104490 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68114460 con 0x7fdf68105d70 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf70600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68114760 con 0x7fdf68106710 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45628/0 (socket says 172.21.15.7:45628) 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf6dc00640 1 -- 172.21.15.7:0/3202003885 learned_addr learned my addr 172.21.15.7:0/3202003885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf6e600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 0x7fdf6819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.245+0000 7fdf6d200640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 -- 172.21.15.7:0/3202003885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 msgr2=0x7fdf6819a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 0x7fdf6819a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 -- 172.21.15.7:0/3202003885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 msgr2=0x7fdf681a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 -- 172.21.15.7:0/3202003885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6819b020 con 0x7fdf68104490 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6e600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 0x7fdf6819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6d200640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.246+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.247+0000 7fdf66000640 1 -- 172.21.15.7:0/3202003885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c00cd80 con 0x7fdf68104490 2024-04-20T21:42:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.247+0000 7fdf66000640 1 -- 172.21.15.7:0/3202003885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf5c010910 con 0x7fdf68104490 2024-04-20T21:42:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.247+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6819b310 con 0x7fdf68104490 2024-04-20T21:42:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.247+0000 7fdf66000640 1 -- 172.21.15.7:0/3202003885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c018990 con 0x7fdf68104490 2024-04-20T21:42:39.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.247+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681a75b0 con 0x7fdf68104490 2024-04-20T21:42:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.250+0000 7fdf733c0740 1 Processor -- start 2024-04-20T21:42:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.250+0000 7fdf733c0740 1 -- start start 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf66000640 1 -- 172.21.15.7:0/3202003885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fdf5c010ac0 con 0x7fdf68104490 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bfa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e851b57150 con 0x55e851bf11a0 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e851afd370 con 0x55e851bf1ac0 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf733c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e851b53110 con 0x55e851bfa6a0 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bfa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf6e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58422/0 (socket says 172.21.15.7:58422) 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.251+0000 7fdf6d200640 1 -- 172.21.15.7:0/1990658074 learned_addr learned my addr 172.21.15.7:0/1990658074 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 -- 172.21.15.7:0/1990658074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 msgr2=0x55e851bfa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bfa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 -- 172.21.15.7:0/1990658074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 msgr2=0x55e851bf1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 -- 172.21.15.7:0/1990658074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8519337d0 con 0x55e851bfa6a0 2024-04-20T21:42:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009860 tx=0x7fdf540056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2541eb8780bfe88 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf5aa00640 1 -- 172.21.15.7:0/1990658074 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54044020 con 0x55e851bfa6a0 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf5aa00640 1 -- 172.21.15.7:0/1990658074 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54004240 con 0x55e851bfa6a0 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.252+0000 7fdf5aa00640 1 -- 172.21.15.7:0/1990658074 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54039040 con 0x55e851bfa6a0 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf66000640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 0x7fdf500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf66000640 1 -- 172.21.15.7:0/3202003885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fdf5c0d0b80 con 0x7fdf68104490 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf6e600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 0x7fdf500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 -- 172.21.15.7:0/1990658074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 msgr2=0x55e851bfca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf54009860 tx=0x7fdf540056d0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 -- 172.21.15.7:0/1990658074 shutdown_connections 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bfa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bfca90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1990658074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 -- 172.21.15.7:0/1990658074 >> 172.21.15.7:0/1990658074 conn(0x55e851beb8d0 msgr2=0x55e851bebcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.253+0000 7fdf733c0740 1 -- 172.21.15.7:0/1990658074 shutdown_connections 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.254+0000 7fdf733c0740 1 -- 172.21.15.7:0/1990658074 wait complete. 2024-04-20T21:42:39.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.254+0000 7fdf6e600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 0x7fdf500a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005fd0 tx=0x7fdf60010040 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.254+0000 7fdf733c0740 1 -- start start 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.254+0000 7fdf733c0740 1 Processor -- start 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e851bfd430 con 0x55e851bf11a0 2024-04-20T21:42:39.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e851b10740 con 0x55e851bf1ac0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e851bfd5b0 con 0x55e851bfa6a0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf733c0740 10 client.0 _ll_register_callbacks cb 0x55e851b0bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:43062/0 (socket says 172.21.15.7:43062) 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 -- 172.21.15.7:0/2401097890 learned_addr learned my addr 172.21.15.7:0/2401097890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6e600640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 -- 172.21.15.7:0/2401097890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 msgr2=0x55e851bf8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bf8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 -- 172.21.15.7:0/2401097890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 msgr2=0x55e851bf76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.255+0000 7fdf6d200640 1 -- 172.21.15.7:0/2401097890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8518693f0 con 0x55e851bf1ac0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf6d200640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf54037040 tx=0x7fdf5402c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=addaa05ea1457656 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf4f400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54050070 con 0x55e851bf1ac0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf54004240 con 0x55e851bf1ac0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fdf54034640 con 0x55e851bf1ac0 2024-04-20T21:42:39.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf5405c050 con 0x55e851bf1ac0 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf733c0740 10 client.27062 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.256+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e8518290f0 con 0x55e851bf1ac0 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54004240 con 0x55e851bf1ac0 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdf54049450 con 0x55e851bf1ac0 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 10 client.27062 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 20 client.27062 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:39.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c0a8a0 0x55e851c0cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 4 client.27062 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e851b64b70 con 0x55e851c0a8a0 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.257+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c0a8a0 0x55e851c0cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.258+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c0a8a0 0x55e851c0cc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e123c3c0e185d037 server_cookie=525d82cc28f78e0d in_seq=0 out_seq=0 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.258+0000 7fdf4f400640 10 client.27062 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf4f400640 1 -- 172.21.15.7:0/2401097890 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e851b64b70 con 0x55e851c0a8a0 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf4f400640 10 client.27062 handle_command_reply: tid=0 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 client.27062 shutdown 2024-04-20T21:42:39.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 msgr2=0x55e851bf7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf54037040 tx=0x7fdf5402c620 comp rx=0 tx=0).stop 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 shutdown_connections 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf1ac0 0x55e851bf7c30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c0a8a0 0x55e851c0cc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bfa6a0 0x55e851bf8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 --2- 172.21.15.7:0/2401097890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851bf76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 >> 172.21.15.7:0/2401097890 conn(0x55e851beb8d0 msgr2=0x55e851bed620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf4f400640 0 client.27062 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 shutdown_connections 2024-04-20T21:42:39.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 1 -- 172.21.15.7:0/2401097890 wait complete. 2024-04-20T21:42:39.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.259+0000 7fdf733c0740 20 client.27062 trim_cache size 0 max 16384 2024-04-20T21:42:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.278+0000 7fdf733c0740 1 Processor -- start 2024-04-20T21:42:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.278+0000 7fdf733c0740 1 -- start start 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf11a0 0x55e851bf9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bfa6a0 0x55e851c06320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681b2710 con 0x55e851bfa6a0 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681affd0 con 0x55e851bf11a0 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf733c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681ade40 con 0x55e851bf1ac0 2024-04-20T21:42:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:58444/0 (socket says 172.21.15.7:58444) 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 -- 172.21.15.7:0/1038535776 learned_addr learned my addr 172.21.15.7:0/1038535776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf11a0 0x55e851bf9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6e600640 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bfa6a0 0x55e851c06320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 -- 172.21.15.7:0/1038535776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf11a0 msgr2=0x55e851bf9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf11a0 0x55e851bf9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 -- 172.21.15.7:0/1038535776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bfa6a0 msgr2=0x55e851c06320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.279+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bfa6a0 0x55e851c06320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.280+0000 7fdf6d200640 1 -- 172.21.15.7:0/1038535776 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e851bfd590 con 0x55e851bf1ac0 2024-04-20T21:42:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.280+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf5402c5f0 tx=0x7fdf540056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc15070260b429b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.280+0000 7fdf5a000640 1 -- 172.21.15.7:0/1038535776 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54050040 con 0x55e851bf1ac0 2024-04-20T21:42:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.280+0000 7fdf5a000640 1 -- 172.21.15.7:0/1038535776 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54032340 con 0x55e851bf1ac0 2024-04-20T21:42:39.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.280+0000 7fdf5a000640 1 -- 172.21.15.7:0/1038535776 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54044040 con 0x55e851bf1ac0 2024-04-20T21:42:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- 172.21.15.7:0/1038535776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 msgr2=0x55e851bf6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdf5402c5f0 tx=0x7fdf540056d0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- 172.21.15.7:0/1038535776 shutdown_connections 2024-04-20T21:42:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bfa6a0 0x55e851c06320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851bf6630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1038535776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bf11a0 0x55e851bf9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- 172.21.15.7:0/1038535776 >> 172.21.15.7:0/1038535776 conn(0x55e851beb8d0 msgr2=0x55e851bea8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- 172.21.15.7:0/1038535776 shutdown_connections 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- 172.21.15.7:0/1038535776 wait complete. 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.281+0000 7fdf733c0740 1 -- start start 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 Processor -- start 2024-04-20T21:42:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851c02d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 0x55e851bff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681ae490 con 0x55e851bf11a0 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681af900 con 0x55e851bfa6a0 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681aeb60 con 0x55e851bf1ac0 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf733c0740 10 client.0 _ll_register_callbacks cb 0x7fdf6000e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.282+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45654/0 (socket says 172.21.15.7:45654) 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 -- 172.21.15.7:0/1718966507 learned_addr learned my addr 172.21.15.7:0/1718966507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:42:39.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851c02d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6e600640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 0x55e851bff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 -- 172.21.15.7:0/1718966507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 msgr2=0x55e851c02d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851c02d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 -- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 msgr2=0x55e851bff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 0x55e851bff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 -- 172.21.15.7:0/1718966507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e851c00000 con 0x55e851bf11a0 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6e600640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 0x55e851bff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851c02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:42:39.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf6dc00640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c09c890 tx=0x7fdf5c0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2f5b5d5f74026a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf4f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.283+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c09c4f0 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf54032340 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fdf5c09ae90 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf5c09cb90 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 10 client.16857 fetch_fsmap learned FSMap version 27 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e851c71630 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54032340 con 0x55e851bf11a0 2024-04-20T21:42:39.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fdf5c018b30 con 0x55e851bf11a0 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 10 client.16857 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 20 client.16857 populate_metadata read hostname 'smithi007' 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c71d00 0x55e851c74120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 4 client.16857 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.284+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e851b5e9e0 con 0x55e851c71d00 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c71d00 0x55e851c74120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf6d200640 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c71d00 0x55e851c74120 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc68a92e04f0eb3 server_cookie=d94d59f465678d90 in_seq=0 out_seq=0 2024-04-20T21:42:39.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf4f400640 10 client.16857 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf4f400640 1 -- 172.21.15.7:0/1718966507 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e851b5e9e0 con 0x55e851c71d00 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf4f400640 10 client.16857 handle_command_reply: tid=0 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf733c0740 1 client.16857 shutdown 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 msgr2=0x55e851c02830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.285+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c09c890 tx=0x7fdf5c0bd040 comp rx=0 tx=0).stop 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 shutdown_connections 2024-04-20T21:42:39.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e851bfa6a0 0x55e851bff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e851c71d00 0x55e851c74120 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e851bf1ac0 0x55e851c02d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 --2- 172.21.15.7:0/1718966507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e851bf11a0 0x55e851c02830 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf4f400640 0 client.16857 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 >> 172.21.15.7:0/1718966507 conn(0x55e851beb8d0 msgr2=0x55e851bf7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 shutdown_connections 2024-04-20T21:42:39.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 1 -- 172.21.15.7:0/1718966507 wait complete. 2024-04-20T21:42:39.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf733c0740 20 client.16857 trim_cache size 0 max 16384 2024-04-20T21:42:39.293 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:42:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:42:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:42:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "cc61036b-02ff-4119-8372-c1473e81cb55": { 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "cc61036b-02ff-4119-8372-c1473e81cb55", 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:42:39.294 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:42:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 msgr2=0x7fdf500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 0x7fdf500a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005fd0 tx=0x7fdf60010040 comp rx=0 tx=0).stop 2024-04-20T21:42:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.286+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 msgr2=0x7fdf681a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:42:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00e9f0 tx=0x7fdf5c00eeb0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 shutdown_connections 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fdf500a3d20 0x7fdf500a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf68105d70 0x7fdf681a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf68106710 0x7fdf6819a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 --2- 172.21.15.7:0/3202003885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf68104490 0x7fdf681a06f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 >> 172.21.15.7:0/3202003885 conn(0x7fdf680ffbf0 msgr2=0x7fdf68100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 shutdown_connections 2024-04-20T21:42:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:42:39.287+0000 7fdf70600640 1 -- 172.21.15.7:0/3202003885 wait complete. 2024-04-20T21:42:39.303 INFO:tasks.cephfs.filesystem:scrub status for tag:cc61036b-02ff-4119-8372-c1473e81cb55 - {'path': '/', 'tag': 'cc61036b-02ff-4119-8372-c1473e81cb55', 'options': 'recursive,force'} 2024-04-20T21:42:39.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:42:39.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:39 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:42:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:42:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:39 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:40.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:42:40.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:39 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:42:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:40 smithi016 ceph-mon[21409]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:40 smithi186 ceph-mon[22157]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:41.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:40 smithi007 ceph-mon[16867]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:42:41.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:41 smithi016 ceph-mon[21409]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:41.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:41 smithi186 ceph-mon[22157]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:42.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:41 smithi007 ceph-mon[16867]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:44.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:44 smithi016 ceph-mon[21409]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:42:44.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:44 smithi186 ceph-mon[22157]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:42:44.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:44 smithi007 ceph-mon[16867]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-20T21:42:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:46 smithi016 ceph-mon[21409]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:46.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:46 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:46 smithi186 ceph-mon[22157]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:46 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:46.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:46 smithi007 ceph-mon[16867]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:46.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:46 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:42:47.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:47 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:47.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:47 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:47.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:47 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:47.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:47 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:47.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:47 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:42:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:47 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:47 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:47 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:47 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:47.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:47 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:42:47.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:47 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:47 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:47 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:42:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:47 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:42:47.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:47 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:42:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:48 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:42:48.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:48 smithi016 ceph-mon[21409]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:48 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:42:48.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:48 smithi186 ceph-mon[22157]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:48.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:48 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:42:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:48 smithi007 ceph-mon[16867]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:50.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:50 smithi016 ceph-mon[21409]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:42:50.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:50 smithi186 ceph-mon[22157]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:42:50.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:50 smithi007 ceph-mon[16867]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-04-20T21:42:52.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:52 smithi016 ceph-mon[21409]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:52.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:52 smithi186 ceph-mon[22157]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:52.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:52 smithi007 ceph-mon[16867]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2024-04-20T21:42:54.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:54 smithi016 ceph-mon[21409]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:54.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:54 smithi186 ceph-mon[22157]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:54.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:54 smithi007 ceph-mon[16867]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-20T21:42:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:56 smithi016 ceph-mon[21409]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-04-20T21:42:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:56 smithi186 ceph-mon[22157]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-04-20T21:42:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:56 smithi007 ceph-mon[16867]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-04-20T21:42:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:42:58 smithi016 ceph-mon[21409]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:42:58 smithi186 ceph-mon[22157]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:42:58.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:42:58 smithi007 ceph-mon[16867]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:43:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:00 smithi016 ceph-mon[21409]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:43:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:00 smithi186 ceph-mon[22157]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:43:00.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:00 smithi007 ceph-mon[16867]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:43:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:02 smithi016 ceph-mon[21409]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:43:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:02 smithi186 ceph-mon[22157]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:43:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:02 smithi007 ceph-mon[16867]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:43:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:04 smithi186 ceph-mon[22157]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:43:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:04 smithi007 ceph-mon[16867]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:43:04.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:04 smithi016 ceph-mon[21409]: pgmap v301: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-20T21:43:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:06 smithi186 ceph-mon[22157]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:43:06.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:06 smithi007 ceph-mon[16867]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:43:06.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:06 smithi016 ceph-mon[21409]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:43:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:08 smithi186 ceph-mon[22157]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:43:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:08 smithi186 ceph-mon[22157]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:43:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:08 smithi007 ceph-mon[16867]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:43:08.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:08 smithi007 ceph-mon[16867]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:43:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:08 smithi016 ceph-mon[21409]: pgmap v303: 97 pgs: 97 active+clean; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:43:08.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:08 smithi016 ceph-mon[21409]: osdmap e96: 12 total, 12 up, 12 in 2024-04-20T21:43:09.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:09.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.606+0000 7f9464400640 1 -- 172.21.15.7:0/3383532693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c1106b0 msgr2=0x7f945c10e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.606+0000 7f9464400640 1 --2- 172.21.15.7:0/3383532693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c1106b0 0x7f945c10e270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9450009900 tx=0x7f945002bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.607+0000 7f9464400640 1 -- 172.21.15.7:0/3383532693 shutdown_connections 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.607+0000 7f9464400640 1 --2- 172.21.15.7:0/3383532693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c11b740 0x7f945c11db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.607+0000 7f9464400640 1 --2- 172.21.15.7:0/3383532693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 0x7f945c10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.607+0000 7f9464400640 1 --2- 172.21.15.7:0/3383532693 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c1106b0 0x7f945c10e270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.607+0000 7f9464400640 1 -- 172.21.15.7:0/3383532693 >> 172.21.15.7:0/3383532693 conn(0x7f945c06d000 msgr2=0x7f945c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.608+0000 7f9464400640 1 -- 172.21.15.7:0/3383532693 shutdown_connections 2024-04-20T21:43:09.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.608+0000 7f9464400640 1 -- 172.21.15.7:0/3383532693 wait complete. 2024-04-20T21:43:09.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.610+0000 7f9464400640 1 Processor -- start 2024-04-20T21:43:09.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.610+0000 7f9464400640 1 -- start start 2024-04-20T21:43:09.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c1106b0 0x7f945c1a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 0x7f945c1a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f945c121140 con 0x7f945c110b20 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c120fc0 con 0x7f945c1106b0 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9464400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1212c0 con 0x7f945c11b740 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9462400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9462400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46218/0 (socket says 172.21.15.7:46218) 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.611+0000 7f9462400640 1 -- 172.21.15.7:0/2401018830 learned_addr learned my addr 172.21.15.7:0/2401018830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:09.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9461000640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 0x7f945c1a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9461a00640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c1106b0 0x7f945c1a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9462400640 1 -- 172.21.15.7:0/2401018830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c1106b0 msgr2=0x7f945c1a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9462400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c1106b0 0x7f945c1a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9462400640 1 -- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 msgr2=0x7f945c1a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9462400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 0x7f945c1a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.612+0000 7f9462400640 1 -- 172.21.15.7:0/2401018830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c1ac580 con 0x7f945c11b740 2024-04-20T21:43:09.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.614+0000 7f9462400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f945400ea00 tx=0x7f945400eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.616+0000 7f945a000640 1 -- 172.21.15.7:0/2401018830 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945400cd80 con 0x7f945c11b740 2024-04-20T21:43:09.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.616+0000 7f945a000640 1 -- 172.21.15.7:0/2401018830 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9454005020 con 0x7f945c11b740 2024-04-20T21:43:09.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.616+0000 7f945a000640 1 -- 172.21.15.7:0/2401018830 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540116d0 con 0x7f945c11b740 2024-04-20T21:43:09.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.616+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c1ac870 con 0x7f945c11b740 2024-04-20T21:43:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.616+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c110f20 con 0x7f945c11b740 2024-04-20T21:43:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.624+0000 7f94671a6740 1 Processor -- start 2024-04-20T21:43:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.624+0000 7f945a000640 1 -- 172.21.15.7:0/2401018830 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f94540051d0 con 0x7f945c11b740 2024-04-20T21:43:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 -- start start 2024-04-20T21:43:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04645160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac046456a0 0x55ac04647a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac045a2150 con 0x55ac046456a0 2024-04-20T21:43:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac04548370 con 0x55ac0463cac0 2024-04-20T21:43:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.625+0000 7f94671a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ac0459e110 con 0x55ac0463c1a0 2024-04-20T21:43:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.627+0000 7f945a000640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 0x7f94440a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.627+0000 7f945a000640 1 -- 172.21.15.7:0/2401018830 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f94540d0e10 con 0x7f945c11b740 2024-04-20T21:43:09.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.629+0000 7f9461000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46234/0 (socket says 172.21.15.7:46234) 2024-04-20T21:43:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04645160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 -- 172.21.15.7:0/3077465967 learned_addr learned my addr 172.21.15.7:0/3077465967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 -- 172.21.15.7:0/3077465967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 msgr2=0x55ac04645160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04645160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 -- 172.21.15.7:0/3077465967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac046456a0 msgr2=0x55ac04647a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac046456a0 0x55ac04647a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9461000640 1 -- 172.21.15.7:0/3077465967 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac0437e7d0 con 0x55ac0463c1a0 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.630+0000 7f9462400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 0x7f94440a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.631+0000 7f9461000640 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9448009c90 tx=0x7f9448012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb6524e58d5cb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.631+0000 7f944ea00640 1 -- 172.21.15.7:0/3077465967 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944801b640 con 0x55ac0463c1a0 2024-04-20T21:43:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.631+0000 7f944ea00640 1 -- 172.21.15.7:0/3077465967 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9448009e40 con 0x55ac0463c1a0 2024-04-20T21:43:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.631+0000 7f944ea00640 1 -- 172.21.15.7:0/3077465967 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944801a880 con 0x55ac0463c1a0 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.632+0000 7f94671a6740 1 -- 172.21.15.7:0/3077465967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 msgr2=0x55ac0463c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.632+0000 7f94671a6740 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9448009c90 tx=0x7f9448012820 comp rx=0 tx=0).stop 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.633+0000 7f94671a6740 1 -- 172.21.15.7:0/3077465967 shutdown_connections 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.633+0000 7f94671a6740 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04645160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.633+0000 7f94671a6740 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac046456a0 0x55ac04647a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.633+0000 7f94671a6740 1 --2- 172.21.15.7:0/3077465967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463c1a0 0x55ac0463c580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.633+0000 7f94671a6740 1 -- 172.21.15.7:0/3077465967 >> 172.21.15.7:0/3077465967 conn(0x55ac046368d0 msgr2=0x55ac04636cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.634+0000 7f94671a6740 1 -- 172.21.15.7:0/3077465967 shutdown_connections 2024-04-20T21:43:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.634+0000 7f94671a6740 1 -- 172.21.15.7:0/3077465967 wait complete. 2024-04-20T21:43:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.635+0000 7f94671a6740 1 -- start start 2024-04-20T21:43:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.636+0000 7f94671a6740 1 Processor -- start 2024-04-20T21:43:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.636+0000 7f9462400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 0x7f94440a62a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94540a54f0 tx=0x7f945400c7f0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04642760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac046431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ac04648490 con 0x55ac0463c1a0 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac0455b740 con 0x55ac0463cac0 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55ac04648610 con 0x55ac046456a0 2024-04-20T21:43:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.638+0000 7f94671a6740 10 client.0 _ll_register_callbacks cb 0x55ac04556b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39960/0 (socket says 172.21.15.7:39960) 2024-04-20T21:43:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 -- 172.21.15.7:0/1109955290 learned_addr learned my addr 172.21.15.7:0/1109955290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 -- 172.21.15.7:0/1109955290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 msgr2=0x55ac046431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac046431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 -- 172.21.15.7:0/1109955290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 msgr2=0x55ac04642760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04642760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.640+0000 7f9461a00640 1 -- 172.21.15.7:0/1109955290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac042b43f0 con 0x55ac0463cac0 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.641+0000 7f9461a00640 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9450009230 tx=0x7f9450009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a1cf4e37de32c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.641+0000 7f9443400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.641+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450005030 con 0x55ac0463cac0 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.641+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f94500051d0 con 0x55ac0463cac0 2024-04-20T21:43:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.641+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac045afb70 con 0x55ac0463cac0 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.642+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94500053d0 con 0x55ac0463cac0 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.642+0000 7f94671a6740 10 client.27080 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.642+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac042740f0 con 0x55ac0463cac0 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.642+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ac045afb70 con 0x55ac0463cac0 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9450003750 con 0x55ac0463cac0 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 10 client.27080 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 10 client.27080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 20 client.27080 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac04655ab0 0x55ac04657f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 4 client.27080 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac045a99e0 con 0x55ac04655ab0 2024-04-20T21:43:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.645+0000 7f9461000640 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac04655ab0 0x55ac04657f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.649+0000 7f9461000640 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac04655ab0 0x55ac04657f00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedb98d92ab49e66 server_cookie=30913bff5bafcb79 in_seq=0 out_seq=0 2024-04-20T21:43:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.649+0000 7f9443400640 10 client.27080 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.650+0000 7f9443400640 1 -- 172.21.15.7:0/1109955290 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f9448004120 con 0x55ac04655ab0 2024-04-20T21:43:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.650+0000 7f9443400640 10 client.27080 handle_command_reply: tid=0 2024-04-20T21:43:09.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.651+0000 7f94671a6740 1 client.27080 shutdown 2024-04-20T21:43:09.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.655+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 msgr2=0x55ac04642ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9450009230 tx=0x7f9450009880 comp rx=0 tx=0).stop 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 shutdown_connections 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463cac0 0x55ac04642ca0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac04655ab0 0x55ac04657f00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac046431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 --2- 172.21.15.7:0/1109955290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04642760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f9443400640 0 client.27080 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:09.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.656+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 >> 172.21.15.7:0/1109955290 conn(0x55ac046368d0 msgr2=0x55ac04638620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.657+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 shutdown_connections 2024-04-20T21:43:09.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.657+0000 7f94671a6740 1 -- 172.21.15.7:0/1109955290 wait complete. 2024-04-20T21:43:09.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.660+0000 7f94671a6740 20 client.27080 trim_cache size 0 max 16384 2024-04-20T21:43:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 Processor -- start 2024-04-20T21:43:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 -- start start 2024-04-20T21:43:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463c1a0 0x55ac04644e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac04635740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1c7cc0 con 0x55ac0463cac0 2024-04-20T21:43:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1c5580 con 0x55ac0463c1a0 2024-04-20T21:43:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.697+0000 7f94671a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f945c068e10 con 0x55ac046456a0 2024-04-20T21:43:09.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.698+0000 7f9461a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.698+0000 7f9461a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38122/0 (socket says 172.21.15.7:38122) 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.698+0000 7f9461a00640 1 -- 172.21.15.7:0/2600329965 learned_addr learned my addr 172.21.15.7:0/2600329965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461000640 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac04635740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461a00640 1 -- 172.21.15.7:0/2600329965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 msgr2=0x55ac04635740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461a00640 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac04635740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461a00640 1 -- 172.21.15.7:0/2600329965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463c1a0 msgr2=0x55ac04644e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461a00640 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463c1a0 0x55ac04644e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461a00640 1 -- 172.21.15.7:0/2600329965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac04630d10 con 0x55ac0463cac0 2024-04-20T21:43:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.699+0000 7f9461000640 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac04635740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.700+0000 7f9461a00640 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9450009850 tx=0x7f945002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d466af4371619e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.700+0000 7f944e000640 1 -- 172.21.15.7:0/2600329965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450004450 con 0x55ac0463cac0 2024-04-20T21:43:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.701+0000 7f944e000640 1 -- 172.21.15.7:0/2600329965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9450057020 con 0x55ac0463cac0 2024-04-20T21:43:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.701+0000 7f944e000640 1 -- 172.21.15.7:0/2600329965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500345e0 con 0x55ac0463cac0 2024-04-20T21:43:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 -- 172.21.15.7:0/2600329965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 msgr2=0x55ac04651350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9450009850 tx=0x7f945002bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 -- 172.21.15.7:0/2600329965 shutdown_connections 2024-04-20T21:43:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac046456a0 0x55ac04635740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463cac0 0x55ac04651350 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 --2- 172.21.15.7:0/2600329965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac0463c1a0 0x55ac04644e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.702+0000 7f94671a6740 1 -- 172.21.15.7:0/2600329965 >> 172.21.15.7:0/2600329965 conn(0x55ac046368d0 msgr2=0x55ac04635030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.703+0000 7f94671a6740 1 -- 172.21.15.7:0/2600329965 shutdown_connections 2024-04-20T21:43:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.703+0000 7f94671a6740 1 -- 172.21.15.7:0/2600329965 wait complete. 2024-04-20T21:43:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.703+0000 7f94671a6740 1 -- start start 2024-04-20T21:43:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.704+0000 7f94671a6740 1 Processor -- start 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04649470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463cac0 0x55ac046499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f945c118070 con 0x55ac0463c1a0 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1194e0 con 0x55ac046456a0 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f945c118740 con 0x55ac0463cac0 2024-04-20T21:43:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.705+0000 7f94671a6740 10 client.0 _ll_register_callbacks cb 0x7f9450009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:09.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.706+0000 7f9461000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04649470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.706+0000 7f9461000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.706+0000 7f9461000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:39984/0 (socket says 172.21.15.7:39984) 2024-04-20T21:43:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.707+0000 7f9461000640 1 -- 172.21.15.7:0/1611490006 learned_addr learned my addr 172.21.15.7:0/1611490006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.707+0000 7f9461a00640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463cac0 0x55ac046499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 -- 172.21.15.7:0/1611490006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463cac0 msgr2=0x55ac046499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463cac0 0x55ac046499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 -- 172.21.15.7:0/1611490006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 msgr2=0x55ac04649470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04649470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 -- 172.21.15.7:0/1611490006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac046bb240 con 0x55ac046456a0 2024-04-20T21:43:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9461000640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9448009c30 tx=0x7f944801a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fc85011d0e2161c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.708+0000 7f9443400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.709+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448005600 con 0x55ac046456a0 2024-04-20T21:43:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.709+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f9448020040 con 0x55ac046456a0 2024-04-20T21:43:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.709+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9450057020 con 0x55ac046456a0 2024-04-20T21:43:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.711+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9448038050 con 0x55ac046456a0 2024-04-20T21:43:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 10 client.27083 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac046bc4a0 con 0x55ac046456a0 2024-04-20T21:43:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9450057020 con 0x55ac046456a0 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f9448019070 con 0x55ac046456a0 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 10 client.27083 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 10 client.27083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 20 client.27083 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac046bca10 0x55ac046bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:09.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 4 client.27083 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.712+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac04525450 con 0x55ac046bca10 2024-04-20T21:43:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.713+0000 7f9461a00640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac046bca10 0x55ac046bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:09.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.714+0000 7f9443400640 10 client.27083 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.714+0000 7f9461a00640 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac046bca10 0x55ac046bee30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b183d643c73b83 server_cookie=787a50a2eaa07bf1 in_seq=0 out_seq=0 2024-04-20T21:43:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.714+0000 7f9443400640 1 -- 172.21.15.7:0/1611490006 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac04525450 con 0x55ac046bca10 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.714+0000 7f9443400640 10 client.27083 handle_command_reply: tid=0 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.714+0000 7f94671a6740 1 client.27083 shutdown 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 msgr2=0x55ac04649ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9448009c30 tx=0x7f944801a480 comp rx=0 tx=0).stop 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 shutdown_connections 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac046456a0 0x55ac04649ef0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55ac046bca10 0x55ac046bee30 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac0463cac0 0x55ac046499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 --2- 172.21.15.7:0/1611490006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ac0463c1a0 0x55ac04649470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f9443400640 0 client.27083 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 >> 172.21.15.7:0/1611490006 conn(0x55ac046368d0 msgr2=0x55ac046384b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 shutdown_connections 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 1 -- 172.21.15.7:0/1611490006 wait complete. 2024-04-20T21:43:09.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.715+0000 7f94671a6740 20 client.27083 trim_cache size 0 max 16384 2024-04-20T21:43:09.720 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:09.720 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:43:09.720 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:43:09.720 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:09.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.716+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 msgr2=0x7f94440a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.716+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 0x7f94440a62a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94540a54f0 tx=0x7f945400c7f0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 msgr2=0x7f945c1abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f945400ea00 tx=0x7f945400eec0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 shutdown_connections 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f945c11b740 0x7f945c1abe80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f945c110b20 0x7f945c1a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f94440a3de0 0x7f94440a62a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 --2- 172.21.15.7:0/2401018830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f945c1106b0 0x7f945c1a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:09.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.717+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 >> 172.21.15.7:0/2401018830 conn(0x7f945c06d000 msgr2=0x7f945c11a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:09.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.718+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 shutdown_connections 2024-04-20T21:43:09.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:09.718+0000 7f9464400640 1 -- 172.21.15.7:0/2401018830 wait complete. 2024-04-20T21:43:09.751 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:43:09.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 -- 172.21.15.7:0/1539419724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c109af0 msgr2=0x7f614c11aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 --2- 172.21.15.7:0/1539419724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c109af0 0x7f614c11aeb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f614001a910 tx=0x7f614003cc60 comp rx=0 tx=0).stop 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 -- 172.21.15.7:0/1539419724 shutdown_connections 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 --2- 172.21.15.7:0/1539419724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c109af0 0x7f614c11aeb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 --2- 172.21.15.7:0/1539419724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c11b480 0x7f614c11d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 --2- 172.21.15.7:0/1539419724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c1091d0 0x7f614c1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 -- 172.21.15.7:0/1539419724 >> 172.21.15.7:0/1539419724 conn(0x7f614c06bf50 msgr2=0x7f614c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.052+0000 7f6153e00640 1 -- 172.21.15.7:0/1539419724 shutdown_connections 2024-04-20T21:43:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.053+0000 7f6153e00640 1 -- 172.21.15.7:0/1539419724 wait complete. 2024-04-20T21:43:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.057+0000 7f6153e00640 1 Processor -- start 2024-04-20T21:43:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.057+0000 7f6153e00640 1 -- start start 2024-04-20T21:43:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 0x7f614c116d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 0x7f614c117770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f614c113090 con 0x7f614c11b480 2024-04-20T21:43:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6151400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6151400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46268/0 (socket says 172.21.15.7:46268) 2024-04-20T21:43:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6151400640 1 -- 172.21.15.7:0/2603457343 learned_addr learned my addr 172.21.15.7:0/2603457343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6151e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 0x7f614c117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f614c112f10 con 0x7f614c1172c0 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.058+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f614c113210 con 0x7f614c1091d0 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.059+0000 7f6150a00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 0x7f614c116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.060+0000 7f6151400640 1 -- 172.21.15.7:0/2603457343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 msgr2=0x7f614c117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151400640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 0x7f614c117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151400640 1 -- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 msgr2=0x7f614c116d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151400640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 0x7f614c116d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151400640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c1ab6b0 con 0x7f614c1091d0 2024-04-20T21:43:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6150a00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 0x7f614c116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 0x7f614c117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.062+0000 7f6151400640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f614400ba50 tx=0x7f614400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.063+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144019070 con 0x7f614c1091d0 2024-04-20T21:43:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.063+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c1ab9a0 con 0x7f614c1091d0 2024-04-20T21:43:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.063+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f614400cbe0 con 0x7f614c1091d0 2024-04-20T21:43:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.063+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440076e0 con 0x7f614c1091d0 2024-04-20T21:43:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.063+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c1abf90 con 0x7f614c1091d0 2024-04-20T21:43:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.065+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c06a3e0 con 0x7f614c1091d0 2024-04-20T21:43:10.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.070+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f614401c070 con 0x7f614c1091d0 2024-04-20T21:43:10.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.074+0000 7f614a000640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 0x7f61340a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.074+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f61440cfd00 con 0x7f614c1091d0 2024-04-20T21:43:10.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.074+0000 7f6150a00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 0x7f61340a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.076+0000 7f6150a00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 0x7f61340a62e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6140019000 tx=0x7f6140000bc0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.078+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f614409c470 con 0x7f614c1091d0 2024-04-20T21:43:10.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.567+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f614c1ac2d0 con 0x7f614c1091d0 2024-04-20T21:43:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.569+0000 7f614a000640 1 -- 172.21.15.7:0/2603457343 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7f614409bd70 con 0x7f614c1091d0 2024-04-20T21:43:10.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:43:10.570 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:43:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:43:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 msgr2=0x7f61340a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 0x7f61340a62e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6140019000 tx=0x7f6140000bc0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 msgr2=0x7f614c11acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f614400ba50 tx=0x7f614400bf10 comp rx=0 tx=0).stop 2024-04-20T21:43:10.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 shutdown_connections 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61340a3e20 0x7f61340a62e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f614c1172c0 0x7f614c117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f614c11b480 0x7f614c116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 --2- 172.21.15.7:0/2603457343 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f614c1091d0 0x7f614c11acf0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.573+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 >> 172.21.15.7:0/2603457343 conn(0x7f614c06bf50 msgr2=0x7f614c06eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.574+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 shutdown_connections 2024-04-20T21:43:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.574+0000 7f6153e00640 1 -- 172.21.15.7:0/2603457343 wait complete. 2024-04-20T21:43:10.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:43:10.787 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:10 smithi007 ceph-mon[16867]: pgmap v305: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:10.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.909+0000 7fae4d600640 1 -- 172.21.15.7:0/2639011268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 msgr2=0x7fae48103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.909+0000 7fae4d600640 1 --2- 172.21.15.7:0/2639011268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48103320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae38009860 tx=0x7fae3802bc00 comp rx=0 tx=0).stop 2024-04-20T21:43:10.948 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:10 smithi016 ceph-mon[21409]: pgmap v305: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 -- 172.21.15.7:0/2639011268 shutdown_connections 2024-04-20T21:43:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 --2- 172.21.15.7:0/2639011268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 0x7fae4810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 --2- 172.21.15.7:0/2639011268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae4810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 --2- 172.21.15.7:0/2639011268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48103320 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 -- 172.21.15.7:0/2639011268 >> 172.21.15.7:0/2639011268 conn(0x7fae48077a10 msgr2=0x7fae480fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.911+0000 7fae4d600640 1 -- 172.21.15.7:0/2639011268 shutdown_connections 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.912+0000 7fae4d600640 1 -- 172.21.15.7:0/2639011268 wait complete. 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.913+0000 7fae4d600640 1 Processor -- start 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.914+0000 7fae4d600640 1 -- start start 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.914+0000 7fae4d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.914+0000 7fae4d600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.914+0000 7fae4d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 0x7fae481114f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.914+0000 7fae47400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.915+0000 7fae47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.915+0000 7fae47400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46286/0 (socket says 172.21.15.7:46286) 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.915+0000 7fae47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40024/0 (socket says 172.21.15.7:40024) 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 0x7fae481114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae4d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae4810fee0 con 0x7fae4810c360 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 -- 172.21.15.7:0/1884223540 learned_addr learned my addr 172.21.15.7:0/1884223540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae4810fd60 con 0x7fae48102f40 2024-04-20T21:43:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fae48110060 con 0x7fae48103860 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 -- 172.21.15.7:0/1884223540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 msgr2=0x7fae48116390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 -- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 msgr2=0x7fae481114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 0x7fae481114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47400640 1 -- 172.21.15.7:0/1884223540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae48111ab0 con 0x7fae48103860 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.916+0000 7fae47e00640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae47400640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c920 tx=0x7fae3400cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae44c00640 1 -- 172.21.15.7:0/1884223540 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007c70 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae44c00640 1 -- 172.21.15.7:0/1884223540 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae34007e10 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae48111da0 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae481b40f0 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.917+0000 7fae44c00640 1 -- 172.21.15.7:0/1884223540 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34018870 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.934+0000 7fae44c00640 1 -- 172.21.15.7:0/1884223540 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fae34018b10 con 0x7fae48103860 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.934+0000 7fae519aa740 1 Processor -- start 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 -- start start 2024-04-20T21:43:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e37550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e40130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5562c3d9c050 con 0x5562c3e37a90 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5562c3d42490 con 0x5562c3e40670 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.936+0000 7fae519aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5562c3d98010 con 0x5562c3e37170 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e40130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40036/0 (socket says 172.21.15.7:40036) 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 -- 172.21.15.7:0/1540476686 learned_addr learned my addr 172.21.15.7:0/1540476686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 -- 172.21.15.7:0/1540476686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 msgr2=0x5562c3e37550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e37550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.937+0000 7fae47400640 1 -- 172.21.15.7:0/1540476686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 msgr2=0x5562c3e40130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.938+0000 7fae47400640 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e40130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.938+0000 7fae47400640 1 -- 172.21.15.7:0/1540476686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c3d4a070 con 0x5562c3e40670 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.938+0000 7fae47400640 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae340cddf0 tx=0x7fae340d6cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a005630c034076cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.939+0000 7fae3d200640 1 -- 172.21.15.7:0/1540476686 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae340df070 con 0x5562c3e40670 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.939+0000 7fae44c00640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 0x7fae2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.940+0000 7fae47e00640 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e40130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.940+0000 7fae3d200640 1 -- 172.21.15.7:0/1540476686 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae340d8d60 con 0x5562c3e40670 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.940+0000 7fae3d200640 1 -- 172.21.15.7:0/1540476686 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae340df070 con 0x5562c3e40670 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.940+0000 7fae4cc00640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 0x7fae2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.940+0000 7fae44c00640 1 -- 172.21.15.7:0/1884223540 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7fae340d0760 con 0x7fae48103860 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.941+0000 7fae519aa740 1 -- 172.21.15.7:0/1540476686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 msgr2=0x5562c3e42a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.941+0000 7fae519aa740 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae340cddf0 tx=0x7fae340d6cb0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 -- 172.21.15.7:0/1540476686 shutdown_connections 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e42a60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e40130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 --2- 172.21.15.7:0/1540476686 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e37550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 -- 172.21.15.7:0/1540476686 >> 172.21.15.7:0/1540476686 conn(0x5562c3e318a0 msgr2=0x5562c3e31c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 -- 172.21.15.7:0/1540476686 shutdown_connections 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.943+0000 7fae519aa740 1 -- 172.21.15.7:0/1540476686 wait complete. 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.944+0000 7fae519aa740 1 -- start start 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.944+0000 7fae519aa740 1 Processor -- start 2024-04-20T21:43:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.944+0000 7fae4cc00640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 0x7fae2c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae480fffe0 tx=0x7fae40009600 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.945+0000 7fae519aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e3d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.945+0000 7fae519aa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e3dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae47e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e3d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae47e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e3d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46306/0 (socket says 172.21.15.7:46306) 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae519aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e3e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae47e00640 1 -- 172.21.15.7:0/269155088 learned_addr learned my addr 172.21.15.7:0/269155088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5562c3e43460 con 0x5562c3e37a90 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.946+0000 7fae4cc00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e3e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5562c3d55860 con 0x5562c3e40670 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5562c3e435e0 con 0x5562c3e37170 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae519aa740 10 client.0 _ll_register_callbacks cb 0x5562c3d23fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 -- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 msgr2=0x5562c3e3d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e3d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 -- 172.21.15.7:0/269155088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 msgr2=0x5562c3e3dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e3dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c3d59950 con 0x5562c3e40670 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.947+0000 7fae4cc00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e3e0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fae40012c10 tx=0x7fae40004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abfd42fb507ecccf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.948+0000 7fae31600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.948+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae340d8d60 con 0x5562c3e40670 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.949+0000 7fae31600640 1 -- 172.21.15.7:0/269155088 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40023070 con 0x5562c3e40670 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.949+0000 7fae31600640 1 -- 172.21.15.7:0/269155088 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7fae40012c40 con 0x5562c3e40670 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.949+0000 7fae31600640 1 -- 172.21.15.7:0/269155088 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae4002f050 con 0x5562c3e40670 2024-04-20T21:43:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.949+0000 7fae519aa740 10 client.27098 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.949+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5562c3d27820 con 0x5562c3e40670 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae31600640 1 -- 172.21.15.7:0/269155088 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fae4000f040 con 0x5562c3e40670 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 10 client.27098 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 10 client.27098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 20 client.27098 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3e50350 0x5562c3e52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 4 client.27098 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.951+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562c3da9a70 con 0x5562c3e50350 2024-04-20T21:43:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.952+0000 7fae47e00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3e50350 0x5562c3e52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.953+0000 7fae31600640 10 client.27098 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:10.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.953+0000 7fae47e00640 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3e50350 0x5562c3e52760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35d4b10fe6171924 server_cookie=556ae6be64fb4b3 in_seq=0 out_seq=0 2024-04-20T21:43:10.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:10 smithi186 ceph-mon[22157]: pgmap v305: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.954+0000 7fae31600640 1 -- 172.21.15.7:0/269155088 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562c3da9a70 con 0x5562c3e50350 2024-04-20T21:43:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.954+0000 7fae31600640 10 client.27098 handle_command_reply: tid=0 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.954+0000 7fae519aa740 1 client.27098 shutdown 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.955+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 msgr2=0x5562c3e3e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.955+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e3e0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fae40012c10 tx=0x7fae40004a40 comp rx=0 tx=0).stop 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 shutdown_connections 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3e50350 0x5562c3e52760 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae31600640 0 client.27098 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e40670 0x5562c3e3e0e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37a90 0x5562c3e3dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 --2- 172.21.15.7:0/269155088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e37170 0x5562c3e3d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 >> 172.21.15.7:0/269155088 conn(0x5562c3e318a0 msgr2=0x5562c3e335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 shutdown_connections 2024-04-20T21:43:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 1 -- 172.21.15.7:0/269155088 wait complete. 2024-04-20T21:43:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.956+0000 7fae519aa740 20 client.27098 trim_cache size 0 max 16384 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.990+0000 7fae519aa740 1 Processor -- start 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 -- start start 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:10.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae28011090 con 0x5562c3e37170 2024-04-20T21:43:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800e950 con 0x5562c3e37a90 2024-04-20T21:43:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae519aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800c7a0 con 0x5562c3e40670 2024-04-20T21:43:10.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.991+0000 7fae47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40058/0 (socket says 172.21.15.7:40058) 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46316/0 (socket says 172.21.15.7:46316) 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 -- 172.21.15.7:0/3028377174 learned_addr learned my addr 172.21.15.7:0/3028377174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 -- 172.21.15.7:0/3028377174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 msgr2=0x5562c3e3df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 -- 172.21.15.7:0/3028377174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 msgr2=0x5562c3e4c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e4c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.992+0000 7fae4cc00640 1 -- 172.21.15.7:0/3028377174 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c3e378c0 con 0x5562c3e40670 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.993+0000 7fae47e00640 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.993+0000 7fae4cc00640 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae4001f4d0 tx=0x7fae40020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f28f87140fa012b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:10.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.993+0000 7fae33e00640 1 -- 172.21.15.7:0/3028377174 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40023070 con 0x5562c3e40670 2024-04-20T21:43:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.995+0000 7fae33e00640 1 -- 172.21.15.7:0/3028377174 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae340d8d60 con 0x5562c3e40670 2024-04-20T21:43:10.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.995+0000 7fae33e00640 1 -- 172.21.15.7:0/3028377174 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40023070 con 0x5562c3e40670 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 -- 172.21.15.7:0/3028377174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 msgr2=0x5562c3e58070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fae4001f4d0 tx=0x7fae40020000 comp rx=0 tx=0).stop 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 -- 172.21.15.7:0/3028377174 shutdown_connections 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3e58070 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 --2- 172.21.15.7:0/3028377174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.998+0000 7fae519aa740 1 -- 172.21.15.7:0/3028377174 >> 172.21.15.7:0/3028377174 conn(0x5562c3e318a0 msgr2=0x5562c3e30960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.999+0000 7fae519aa740 1 -- 172.21.15.7:0/3028377174 shutdown_connections 2024-04-20T21:43:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:10.999+0000 7fae519aa740 1 -- 172.21.15.7:0/3028377174 wait complete. 2024-04-20T21:43:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.000+0000 7fae519aa740 1 -- start start 2024-04-20T21:43:11.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.000+0000 7fae519aa740 1 Processor -- start 2024-04-20T21:43:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e3aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800ce10 con 0x5562c3e37170 2024-04-20T21:43:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800e280 con 0x5562c3e37a90 2024-04-20T21:43:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fae2800d4e0 con 0x5562c3e40670 2024-04-20T21:43:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae519aa740 10 client.0 _ll_register_callbacks cb 0x7fae400128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae4cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46326/0 (socket says 172.21.15.7:46326) 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.002+0000 7fae4cc00640 1 -- 172.21.15.7:0/4293070953 learned_addr learned my addr 172.21.15.7:0/4293070953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e3aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 -- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 msgr2=0x5562c3eb58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 -- 172.21.15.7:0/4293070953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 msgr2=0x5562c3e3af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae47e00640 1 -- 172.21.15.7:0/4293070953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c3aae3f0 con 0x5562c3e37170 2024-04-20T21:43:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.003+0000 7fae4cc00640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.004+0000 7fae47e00640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e3aa20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae38000c00 tx=0x7fae380056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2201818d19bb3ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.004+0000 7fae31600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.005+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38039040 con 0x5562c3e37170 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.005+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7fae38034440 con 0x5562c3e37170 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.006+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae340d8d60 con 0x5562c3e37170 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.006+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae38034640 con 0x5562c3e37170 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.006+0000 7fae519aa740 10 client.16884 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.006+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562c3b787d0 con 0x5562c3e37170 2024-04-20T21:43:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.007+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae340d8d60 con 0x5562c3e37170 2024-04-20T21:43:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.007+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7fae3802c9c0 con 0x5562c3e37170 2024-04-20T21:43:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 10 client.16884 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 10 client.16884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:43:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 20 client.16884 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3eb74a0 0x5562c3eb98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 4 client.16884 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562c3da38e0 con 0x5562c3eb74a0 2024-04-20T21:43:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.008+0000 7fae47400640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3eb74a0 0x5562c3eb98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.011+0000 7fae47400640 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3eb74a0 0x5562c3eb98c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cffac3e8f0c0422d server_cookie=863f5ea0bbbfd124 in_seq=0 out_seq=0 2024-04-20T21:43:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.011+0000 7fae31600640 10 client.16884 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae31600640 1 -- 172.21.15.7:0/4293070953 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562c3da38e0 con 0x5562c3eb74a0 2024-04-20T21:43:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae31600640 10 client.16884 handle_command_reply: tid=0 2024-04-20T21:43:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 client.16884 shutdown 2024-04-20T21:43:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 msgr2=0x5562c3e3aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e3aa20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae38000c00 tx=0x7fae380056d0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 shutdown_connections 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x5562c3eb74a0 0x5562c3eb98c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5562c3e37a90 0x5562c3e3af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae31600640 0 client.16884 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5562c3e40670 0x5562c3eb58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 --2- 172.21.15.7:0/4293070953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5562c3e37170 0x5562c3e3aa20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.012+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 >> 172.21.15.7:0/4293070953 conn(0x5562c3e318a0 msgr2=0x5562c3e32a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.013+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 shutdown_connections 2024-04-20T21:43:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.014+0000 7fae519aa740 1 -- 172.21.15.7:0/4293070953 wait complete. 2024-04-20T21:43:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.014+0000 7fae519aa740 20 client.16884 trim_cache size 0 max 16384 2024-04-20T21:43:11.015 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:11.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 msgr2=0x7fae2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 0x7fae2c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae480fffe0 tx=0x7fae40009600 comp rx=0 tx=0).stop 2024-04-20T21:43:11.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 msgr2=0x7fae481168d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c920 tx=0x7fae3400cde0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 shutdown_connections 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae4810c360 0x7fae481114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae48103860 0x7fae481168d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fae2c0a3da0 0x7fae2c0a6260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 --2- 172.21.15.7:0/1884223540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae48102f40 0x7fae48116390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.015+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 >> 172.21.15.7:0/1884223540 conn(0x7fae48077a10 msgr2=0x7fae480fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.016+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 shutdown_connections 2024-04-20T21:43:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.016+0000 7fae4d600640 1 -- 172.21.15.7:0/1884223540 wait complete. 2024-04-20T21:43:11.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:43:11.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.344+0000 7f7c14000640 1 -- 172.21.15.7:0/3291111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c1150b0 msgr2=0x7f7c0c1174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.344+0000 7f7c14000640 1 --2- 172.21.15.7:0/3291111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1174a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009930 tx=0x7f7c0402bc50 comp rx=0 tx=0).stop 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.346+0000 7f7c14000640 1 -- 172.21.15.7:0/3291111704 shutdown_connections 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.346+0000 7f7c14000640 1 --2- 172.21.15.7:0/3291111704 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1174a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.346+0000 7f7c14000640 1 --2- 172.21.15.7:0/3291111704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 0x7f7c0c114b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.346+0000 7f7c14000640 1 --2- 172.21.15.7:0/3291111704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c109140 0x7f7c0c109520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.346+0000 7f7c14000640 1 -- 172.21.15.7:0/3291111704 >> 172.21.15.7:0/3291111704 conn(0x7f7c0c06d000 msgr2=0x7f7c0c06d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.347+0000 7f7c14000640 1 -- 172.21.15.7:0/3291111704 shutdown_connections 2024-04-20T21:43:11.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.348+0000 7f7c14000640 1 -- 172.21.15.7:0/3291111704 wait complete. 2024-04-20T21:43:11.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.349+0000 7f7c14000640 1 Processor -- start 2024-04-20T21:43:11.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.349+0000 7f7c14000640 1 -- start start 2024-04-20T21:43:11.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.350+0000 7f7c14000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c14000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 0x7f7c0c1a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c14000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c14000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c11ac30 con 0x7f7c0c109a60 2024-04-20T21:43:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c14000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c11aab0 con 0x7f7c0c1150b0 2024-04-20T21:43:11.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c14000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c11adb0 con 0x7f7c0c109140 2024-04-20T21:43:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c11600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c11600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46338/0 (socket says 172.21.15.7:46338) 2024-04-20T21:43:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c11600640 1 -- 172.21.15.7:0/1169438334 learned_addr learned my addr 172.21.15.7:0/1169438334 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c12000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.351+0000 7f7c10c00640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 0x7f7c0c1a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.352+0000 7f7c11600640 1 -- 172.21.15.7:0/1169438334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c1150b0 msgr2=0x7f7c0c1a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.352+0000 7f7c11600640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.352+0000 7f7c11600640 1 -- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 msgr2=0x7f7c0c1a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.352+0000 7f7c11600640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 0x7f7c0c1a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.352+0000 7f7c11600640 1 -- 172.21.15.7:0/1169438334 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1a82e0 con 0x7f7c0c109140 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.354+0000 7f7c11600640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b4a0 tx=0x7f7c0000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.354+0000 7f7c0a000640 1 -- 172.21.15.7:0/1169438334 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00014070 con 0x7f7c0c109140 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.354+0000 7f7c0a000640 1 -- 172.21.15.7:0/1169438334 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c00010af0 con 0x7f7c0c109140 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.354+0000 7f7c0a000640 1 -- 172.21.15.7:0/1169438334 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00017d10 con 0x7f7c0c109140 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.354+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1a85d0 con 0x7f7c0c109140 2024-04-20T21:43:11.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.355+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1afe60 con 0x7f7c0c109140 2024-04-20T21:43:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.359+0000 7f7c16e37740 1 Processor -- start 2024-04-20T21:43:11.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.360+0000 7f7c16e37740 1 -- start start 2024-04-20T21:43:11.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.360+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 0x557f863a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 0x557f863aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863b04b0 0x557f863b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c16e37740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f8630c050 con 0x557f863a6fb0 2024-04-20T21:43:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c16e37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f862b2490 con 0x557f863a78d0 2024-04-20T21:43:11.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c16e37740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f86308010 con 0x557f863b04b0 2024-04-20T21:43:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c12000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 0x557f863a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.362+0000 7f7c10c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863b04b0 0x557f863b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.363+0000 7f7c0a000640 1 -- 172.21.15.7:0/1169438334 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f7c00017070 con 0x7f7c0c109140 2024-04-20T21:43:11.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.364+0000 7f7c11600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 0x557f863aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.364+0000 7f7c11600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 0x557f863aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40098/0 (socket says 172.21.15.7:40098) 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.364+0000 7f7c11600640 1 -- 172.21.15.7:0/1134096775 learned_addr learned my addr 172.21.15.7:0/1134096775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.365+0000 7f7c12000640 1 -- 172.21.15.7:0/1134096775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863b04b0 msgr2=0x557f863b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.365+0000 7f7c12000640 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863b04b0 0x557f863b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.365+0000 7f7c12000640 1 -- 172.21.15.7:0/1134096775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 msgr2=0x557f863aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.365+0000 7f7c12000640 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 0x557f863aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.365+0000 7f7c12000640 1 -- 172.21.15.7:0/1134096775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f862ba070 con 0x557f863a6fb0 2024-04-20T21:43:11.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.366+0000 7f7c12000640 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 0x557f863a7390 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402c260 tx=0x7f7c0402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c22889e4de5fe6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.368+0000 7f7bfa600640 1 -- 172.21.15.7:0/1134096775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04042070 con 0x557f863a6fb0 2024-04-20T21:43:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.369+0000 7f7bfa600640 1 -- 172.21.15.7:0/1134096775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04004300 con 0x557f863a6fb0 2024-04-20T21:43:11.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.369+0000 7f7bfa600640 1 -- 172.21.15.7:0/1134096775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04005000 con 0x557f863a6fb0 2024-04-20T21:43:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.369+0000 7f7c0a000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 0x7f7bf40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.369+0000 7f7c0a000640 1 -- 172.21.15.7:0/1169438334 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f7c000cfa80 con 0x7f7c0c109140 2024-04-20T21:43:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 -- 172.21.15.7:0/1134096775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 msgr2=0x557f863a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 0x557f863a7390 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c0402c260 tx=0x7f7c0402c5f0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 -- 172.21.15.7:0/1134096775 shutdown_connections 2024-04-20T21:43:11.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a78d0 0x557f863aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863b04b0 0x557f863b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1134096775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a6fb0 0x557f863a7390 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 -- 172.21.15.7:0/1134096775 >> 172.21.15.7:0/1134096775 conn(0x557f863a16e0 msgr2=0x557f863a1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 -- 172.21.15.7:0/1134096775 shutdown_connections 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.370+0000 7f7c16e37740 1 -- 172.21.15.7:0/1134096775 wait complete. 2024-04-20T21:43:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.371+0000 7f7c11600640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 0x7f7bf40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.372+0000 7f7c11600640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 0x7f7bf40a62e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c000984c0 tx=0x7f7c0000bf90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.372+0000 7f7c16e37740 1 -- start start 2024-04-20T21:43:11.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.372+0000 7f7c16e37740 1 Processor -- start 2024-04-20T21:43:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x557f863b3240 con 0x557f863a78d0 2024-04-20T21:43:11.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557f862c5860 con 0x557f863b04b0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x557f863b33c0 con 0x557f863a6fb0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.373+0000 7f7c16e37740 10 client.0 _ll_register_callbacks cb 0x557f86293fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46356/0 (socket says 172.21.15.7:46356) 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 -- 172.21.15.7:0/1160910706 learned_addr learned my addr 172.21.15.7:0/1160910706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c10c00640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 -- 172.21.15.7:0/1160910706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 msgr2=0x557f863adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 -- 172.21.15.7:0/1160910706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 msgr2=0x557f863adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.374+0000 7f7c12000640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.375+0000 7f7c12000640 1 -- 172.21.15.7:0/1160910706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f862c9950 con 0x557f863a6fb0 2024-04-20T21:43:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.375+0000 7f7c12000640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ec0 tx=0x7f7c0402e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ef8e1b9afa5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.375+0000 7f7beea00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.377+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04042030 con 0x557f863a6fb0 2024-04-20T21:43:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.377+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f7c04005450 con 0x557f863a6fb0 2024-04-20T21:43:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.377+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c04004300 con 0x557f863a6fb0 2024-04-20T21:43:11.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.378+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7c0402c8a0 con 0x557f863a6fb0 2024-04-20T21:43:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.378+0000 7f7c16e37740 10 client.27055 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.378+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f86297820 con 0x557f863a6fb0 2024-04-20T21:43:11.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.378+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04004300 con 0x557f863a6fb0 2024-04-20T21:43:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.380+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f7c04003cb0 con 0x557f863a6fb0 2024-04-20T21:43:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 10 client.27055 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 10 client.27055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:43:11.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 20 client.27055 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f863c0360 0x557f863c2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 4 client.27055 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f86319a70 con 0x557f863c0360 2024-04-20T21:43:11.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.381+0000 7f7c10c00640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f863c0360 0x557f863c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.382+0000 7f7beea00640 10 client.27055 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:11.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.382+0000 7f7c10c00640 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f863c0360 0x557f863c2750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc155e2b4d614fc9 server_cookie=1f0958fdc13e14e9 in_seq=0 out_seq=0 2024-04-20T21:43:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.383+0000 7f7beea00640 1 -- 172.21.15.7:0/1160910706 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f86319a70 con 0x557f863c0360 2024-04-20T21:43:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.383+0000 7f7beea00640 10 client.27055 handle_command_reply: tid=0 2024-04-20T21:43:11.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.384+0000 7f7c16e37740 1 client.27055 shutdown 2024-04-20T21:43:11.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.384+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 msgr2=0x557f863ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.384+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ec0 tx=0x7f7c0402e040 comp rx=0 tx=0).stop 2024-04-20T21:43:11.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 shutdown_connections 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f863c0360 0x557f863c2750 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1160910706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863ad560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 >> 172.21.15.7:0/1160910706 conn(0x557f863a16e0 msgr2=0x557f863a3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.386+0000 7f7beea00640 0 client.27055 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.388+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 shutdown_connections 2024-04-20T21:43:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.388+0000 7f7c16e37740 1 -- 172.21.15.7:0/1160910706 wait complete. 2024-04-20T21:43:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.388+0000 7f7c16e37740 20 client.27055 trim_cache size 0 max 16384 2024-04-20T21:43:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.421+0000 7f7c16e37740 1 Processor -- start 2024-04-20T21:43:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.421+0000 7f7c16e37740 1 -- start start 2024-04-20T21:43:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b6b70 con 0x557f863a78d0 2024-04-20T21:43:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b4430 con 0x557f863b04b0 2024-04-20T21:43:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c16e37740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b22c0 con 0x557f863a6fb0 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c12000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c12000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46368/0 (socket says 172.21.15.7:46368) 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.422+0000 7f7c12000640 1 -- 172.21.15.7:0/1626839648 learned_addr learned my addr 172.21.15.7:0/1626839648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.423+0000 7f7c12000640 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.423+0000 7f7c12000640 1 -- 172.21.15.7:0/1626839648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 msgr2=0x557f863bbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.423+0000 7f7c12000640 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863bbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.424+0000 7f7c12000640 1 -- 172.21.15.7:0/1626839648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 msgr2=0x557f863ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.424+0000 7f7c12000640 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.424+0000 7f7c12000640 1 -- 172.21.15.7:0/1626839648 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f8601e3f0 con 0x557f863a6fb0 2024-04-20T21:43:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.424+0000 7f7c12000640 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ec0 tx=0x7f7c04005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61af65f7fde30e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.426+0000 7f7bf9c00640 1 -- 172.21.15.7:0/1626839648 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04042040 con 0x557f863a6fb0 2024-04-20T21:43:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.426+0000 7f7bf9c00640 1 -- 172.21.15.7:0/1626839648 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04054e10 con 0x557f863a6fb0 2024-04-20T21:43:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.426+0000 7f7bf9c00640 1 -- 172.21.15.7:0/1626839648 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c040463c0 con 0x557f863a6fb0 2024-04-20T21:43:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.427+0000 7f7c16e37740 1 -- 172.21.15.7:0/1626839648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 msgr2=0x557f863afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.427+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7c04005ec0 tx=0x7f7c04005fd0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.428+0000 7f7c16e37740 1 -- 172.21.15.7:0/1626839648 shutdown_connections 2024-04-20T21:43:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.428+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863b04b0 0x557f863bbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.428+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863a78d0 0x557f863ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.428+0000 7f7c16e37740 1 --2- 172.21.15.7:0/1626839648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a6fb0 0x557f863afc90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.428+0000 7f7c16e37740 1 -- 172.21.15.7:0/1626839648 >> 172.21.15.7:0/1626839648 conn(0x557f863a16e0 msgr2=0x557f863bb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.429+0000 7f7c16e37740 1 -- 172.21.15.7:0/1626839648 shutdown_connections 2024-04-20T21:43:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.429+0000 7f7c16e37740 1 -- 172.21.15.7:0/1626839648 wait complete. 2024-04-20T21:43:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.429+0000 7f7c16e37740 1 -- start start 2024-04-20T21:43:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.430+0000 7f7c16e37740 1 Processor -- start 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.430+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 0x557f863c2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 0x557f863c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c12000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 0x557f863c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 0x557f863c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b2910 con 0x557f863b04b0 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b3d60 con 0x557f863a6fb0 2024-04-20T21:43:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c1b2fc0 con 0x557f863a78d0 2024-04-20T21:43:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c12000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 0x557f863c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40128/0 (socket says 172.21.15.7:40128) 2024-04-20T21:43:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c12000640 1 -- 172.21.15.7:0/112989023 learned_addr learned my addr 172.21.15.7:0/112989023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c16e37740 10 client.0 _ll_register_callbacks cb 0x7f7c04004f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.431+0000 7f7c10c00640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 0x557f863c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.432+0000 7f7c12000640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 0x557f863c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.432+0000 7f7c10c00640 1 -- 172.21.15.7:0/112989023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 msgr2=0x557f863c2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.433+0000 7f7c10c00640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 0x557f863c2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.433+0000 7f7c10c00640 1 -- 172.21.15.7:0/112989023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 msgr2=0x557f863c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.433+0000 7f7c10c00640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 0x557f863c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.434+0000 7f7c10c00640 1 -- 172.21.15.7:0/112989023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f860e87d0 con 0x557f863a78d0 2024-04-20T21:43:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.434+0000 7f7c12000640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 0x557f863c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.435+0000 7f7c10c00640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 0x557f863c28b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x557f863b3400 tx=0x7f7bfc007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3f7cbc7d3140d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.435+0000 7f7beea00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.435+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc014690 con 0x557f863a78d0 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.435+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c04054e10 con 0x557f863a78d0 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.436+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f7bfc014830 con 0x557f863a78d0 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.436+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7bfc034050 con 0x557f863a78d0 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.436+0000 7f7c16e37740 10 client.27061 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:11.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.436+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f86425970 con 0x557f863a78d0 2024-04-20T21:43:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.437+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c04054e10 con 0x557f863a78d0 2024-04-20T21:43:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f7bfc0207c0 con 0x557f863a78d0 2024-04-20T21:43:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 10 client.27061 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 10 client.27061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:43:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 20 client.27061 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f86426050 0x557f86428470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 4 client.27061 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f863138e0 con 0x557f86426050 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.439+0000 7f7c12000640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f86426050 0x557f86428470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.440+0000 7f7c12000640 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f86426050 0x557f86428470 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=790220f9117eb758 server_cookie=fc1879b865606d8e in_seq=0 out_seq=0 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.440+0000 7f7beea00640 10 client.27061 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.440+0000 7f7beea00640 1 -- 172.21.15.7:0/112989023 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f863138e0 con 0x557f86426050 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.440+0000 7f7beea00640 10 client.27061 handle_command_reply: tid=0 2024-04-20T21:43:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.440+0000 7f7c16e37740 1 client.27061 shutdown 2024-04-20T21:43:11.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 msgr2=0x557f863c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 0x557f863c28b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x557f863b3400 tx=0x7f7bfc007d30 comp rx=0 tx=0).stop 2024-04-20T21:43:11.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 shutdown_connections 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x557f86426050 0x557f86428470 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557f863b04b0 0x557f863c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x557f863a78d0 0x557f863c28b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 --2- 172.21.15.7:0/112989023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557f863a6fb0 0x557f863c2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 >> 172.21.15.7:0/112989023 conn(0x557f863a16e0 msgr2=0x557f863ac5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7beea00640 0 client.27061 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.441+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 shutdown_connections 2024-04-20T21:43:11.443 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:11.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.442+0000 7f7c16e37740 1 -- 172.21.15.7:0/112989023 wait complete. 2024-04-20T21:43:11.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.442+0000 7f7c16e37740 20 client.27061 trim_cache size 0 max 16384 2024-04-20T21:43:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.444+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 msgr2=0x7f7bf40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.444+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 0x7f7bf40a62e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7c000984c0 tx=0x7f7c0000bf90 comp rx=0 tx=0).stop 2024-04-20T21:43:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.444+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 msgr2=0x7f7c0c1a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.444+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b4a0 tx=0x7f7c0000b960 comp rx=0 tx=0).stop 2024-04-20T21:43:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 shutdown_connections 2024-04-20T21:43:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f7bf40a3e20 0x7f7bf40a62e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0c1150b0 0x7f7c0c1a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c0c109a60 0x7f7c0c1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 --2- 172.21.15.7:0/1169438334 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c0c109140 0x7f7c0c1a32f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 >> 172.21.15.7:0/1169438334 conn(0x7f7c0c06d000 msgr2=0x7f7c0c10a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 shutdown_connections 2024-04-20T21:43:11.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.446+0000 7f7c14000640 1 -- 172.21.15.7:0/1169438334 wait complete. 2024-04-20T21:43:11.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:43:11.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 -- 172.21.15.7:0/1425719796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c4101870 msgr2=0x7f77c41058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 --2- 172.21.15.7:0/1425719796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c4101870 0x7f77c41058b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009900 tx=0x7f77ac02bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 -- 172.21.15.7:0/1425719796 shutdown_connections 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 --2- 172.21.15.7:0/1425719796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4105f90 0x7f77c406a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 --2- 172.21.15.7:0/1425719796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c4101870 0x7f77c41058b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 --2- 172.21.15.7:0/1425719796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.775+0000 7f77c8a00640 1 -- 172.21.15.7:0/1425719796 >> 172.21.15.7:0/1425719796 conn(0x7f77c40fc7a0 msgr2=0x7f77c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.776+0000 7f77c8a00640 1 -- 172.21.15.7:0/1425719796 shutdown_connections 2024-04-20T21:43:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.776+0000 7f77c8a00640 1 -- 172.21.15.7:0/1425719796 wait complete. 2024-04-20T21:43:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 Processor -- start 2024-04-20T21:43:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 -- start start 2024-04-20T21:43:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4105f90 0x7f77c41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c419a710 0x7f77c419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41143a0 con 0x7f77c4105f90 2024-04-20T21:43:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4114220 con 0x7f77c4100ec0 2024-04-20T21:43:11.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.779+0000 7f77c8a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4114520 con 0x7f77c419a710 2024-04-20T21:43:11.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.780+0000 7f77c2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.780+0000 7f77c2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40142/0 (socket says 172.21.15.7:40142) 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.780+0000 7f77c2800640 1 -- 172.21.15.7:0/82442830 learned_addr learned my addr 172.21.15.7:0/82442830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.780+0000 7f77c3200640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c419a710 0x7f77c419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.781+0000 7f77c2800640 1 -- 172.21.15.7:0/82442830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c419a710 msgr2=0x7f77c419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.781+0000 7f77c2800640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c419a710 0x7f77c419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.781+0000 7f77c2800640 1 -- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4105f90 msgr2=0x7f77c41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.781+0000 7f77c2800640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4105f90 0x7f77c41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.781+0000 7f77c2800640 1 -- 172.21.15.7:0/82442830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c419b310 con 0x7f77c4100ec0 2024-04-20T21:43:11.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:11 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2603457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.783+0000 7f77c2800640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f77b400e9f0 tx=0x7f77b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.783+0000 7f77bb400640 1 -- 172.21.15.7:0/82442830 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b400cd80 con 0x7f77c4100ec0 2024-04-20T21:43:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.783+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c410fd70 con 0x7f77c4100ec0 2024-04-20T21:43:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.783+0000 7f77bb400640 1 -- 172.21.15.7:0/82442830 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b4010910 con 0x7f77c4100ec0 2024-04-20T21:43:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.783+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c4110250 con 0x7f77c4100ec0 2024-04-20T21:43:11.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.784+0000 7f77bb400640 1 -- 172.21.15.7:0/82442830 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4018990 con 0x7f77c4100ec0 2024-04-20T21:43:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.790+0000 7f77cc1c2740 1 Processor -- start 2024-04-20T21:43:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.794+0000 7f77cc1c2740 1 -- start start 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038cfe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d00a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e038c5a050 con 0x55e038cf5a90 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e038c00490 con 0x55e038cfe670 2024-04-20T21:43:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e038c56010 con 0x55e038cf5170 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77c3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77c3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46406/0 (socket says 172.21.15.7:46406) 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.795+0000 7f77c3200640 1 -- 172.21.15.7:0/3875322965 learned_addr learned my addr 172.21.15.7:0/3875322965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 -- 172.21.15.7:0/3875322965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 msgr2=0x55e038d00a60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d00a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 -- 172.21.15.7:0/3875322965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 msgr2=0x55e038cfe130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038cfe130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 -- 172.21.15.7:0/3875322965 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e038c08070 con 0x55e038cf5170 2024-04-20T21:43:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77c3200640 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009f70 tx=0x7f77bc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5448e00da7b90974 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.796+0000 7f77b3400640 1 -- 172.21.15.7:0/3875322965 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc016070 con 0x55e038cf5170 2024-04-20T21:43:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.797+0000 7f77bb400640 1 -- 172.21.15.7:0/82442830 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f77b4010ac0 con 0x7f77c4100ec0 2024-04-20T21:43:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.797+0000 7f77b3400640 1 -- 172.21.15.7:0/3875322965 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc004d20 con 0x55e038cf5170 2024-04-20T21:43:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.797+0000 7f77b3400640 1 -- 172.21.15.7:0/3875322965 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0128f0 con 0x55e038cf5170 2024-04-20T21:43:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.800+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3875322965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 msgr2=0x55e038cf5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.800+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009f70 tx=0x7f77bc010800 comp rx=0 tx=0).stop 2024-04-20T21:43:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.800+0000 7f77bb400640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 0x7f77a80a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3875322965 shutdown_connections 2024-04-20T21:43:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d00a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038cfe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3875322965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038cf5550 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3875322965 >> 172.21.15.7:0/3875322965 conn(0x55e038cef8a0 msgr2=0x55e038cefc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77c3200640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 0x7f77a80a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.802+0000 7f77bb400640 1 -- 172.21.15.7:0/82442830 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f77b40d0b70 con 0x7f77c4100ec0 2024-04-20T21:43:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.803+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3875322965 shutdown_connections 2024-04-20T21:43:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.803+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3875322965 wait complete. 2024-04-20T21:43:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.804+0000 7f77cc1c2740 1 -- start start 2024-04-20T21:43:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.804+0000 7f77cc1c2740 1 Processor -- start 2024-04-20T21:43:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.805+0000 7f77c3200640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 0x7f77a80a62e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77bc005b40 tx=0x7f77bc010fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.806+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5170 0x55e038cfb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.806+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 0x55e038cfbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.806+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 0x55e038cfc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.807+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e038d01460 con 0x55e038cf5170 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.807+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e038c13860 con 0x55e038cf5a90 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.807+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55e038d015e0 con 0x55e038cfe670 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.808+0000 7f77cc1c2740 10 client.0 _ll_register_callbacks cb 0x55e038be1fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.807+0000 7f77c3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 0x55e038cfc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.808+0000 7f77c3200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 0x55e038cfc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.808+0000 7f77c3200640 1 -- 172.21.15.7:0/621480092 learned_addr learned my addr 172.21.15.7:0/621480092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.807+0000 7f77bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 0x55e038cfbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.808+0000 7f77c2800640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5170 0x55e038cfb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.809+0000 7f77bbe00640 1 -- 172.21.15.7:0/621480092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 msgr2=0x55e038cfc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.809+0000 7f77bbe00640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 0x55e038cfc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.809+0000 7f77bbe00640 1 -- 172.21.15.7:0/621480092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5170 msgr2=0x55e038cfb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.809+0000 7f77bbe00640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5170 0x55e038cfb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.809+0000 7f77bbe00640 1 -- 172.21.15.7:0/621480092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e038c17950 con 0x55e038cf5a90 2024-04-20T21:43:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.810+0000 7f77bbe00640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 0x55e038cfbbd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e038cf0a30 tx=0x7f77ac0045f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c82e8899afb537 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.810+0000 7f77a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.811+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac042030 con 0x55e038cf5a90 2024-04-20T21:43:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.811+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f77ac02ca90 con 0x55e038cf5a90 2024-04-20T21:43:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.812+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc004d20 con 0x55e038cf5a90 2024-04-20T21:43:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.813+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77ac005d50 con 0x55e038cf5a90 2024-04-20T21:43:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.813+0000 7f77cc1c2740 10 client.27128 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.813+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e038be5820 con 0x55e038cf5a90 2024-04-20T21:43:11.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.813+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77bc004d20 con 0x55e038cf5a90 2024-04-20T21:43:11.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f77ac004b80 con 0x55e038cf5a90 2024-04-20T21:43:11.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 10 client.27128 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 10 client.27128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 20 client.27128 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d0e670 0x55e038d10aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 4 client.27128 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e038c67a70 con 0x55e038d0e670 2024-04-20T21:43:11.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.815+0000 7f77c2800640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d0e670 0x55e038d10aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.817+0000 7f77c2800640 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d0e670 0x55e038d10aa0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac5537b36c2b150 server_cookie=b9d64fa062b4f1b5 in_seq=0 out_seq=0 2024-04-20T21:43:11.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.817+0000 7f77a7e00640 10 client.27128 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.819+0000 7f77a7e00640 1 -- 172.21.15.7:0/621480092 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e038c67a70 con 0x55e038d0e670 2024-04-20T21:43:11.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.819+0000 7f77a7e00640 10 client.27128 handle_command_reply: tid=0 2024-04-20T21:43:11.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.819+0000 7f77cc1c2740 1 client.27128 shutdown 2024-04-20T21:43:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.821+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 msgr2=0x55e038cfbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.821+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 0x55e038cfbbd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55e038cf0a30 tx=0x7f77ac0045f0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 shutdown_connections 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5a90 0x55e038cfbbd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d0e670 0x55e038d10aa0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cfe670 0x55e038cfc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/621480092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5170 0x55e038cfb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.822+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 >> 172.21.15.7:0/621480092 conn(0x55e038cef8a0 msgr2=0x55e038cf15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.824+0000 7f77a7e00640 0 client.27128 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.824+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 shutdown_connections 2024-04-20T21:43:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.825+0000 7f77cc1c2740 1 -- 172.21.15.7:0/621480092 wait complete. 2024-04-20T21:43:11.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.825+0000 7f77cc1c2740 20 client.27128 trim_cache size 0 max 16384 2024-04-20T21:43:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.853+0000 7f77cc1c2740 1 Processor -- start 2024-04-20T21:43:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 -- start start 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5a90 0x55e038cfbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cfe670 0x55e038cf13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b6a90 con 0x55e038cfe670 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b4350 con 0x55e038cf5170 2024-04-20T21:43:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b21e0 con 0x55e038cf5a90 2024-04-20T21:43:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77c2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77c2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40164/0 (socket says 172.21.15.7:40164) 2024-04-20T21:43:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.854+0000 7f77c2800640 1 -- 172.21.15.7:0/1379217614 learned_addr learned my addr 172.21.15.7:0/1379217614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77bbe00640 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5a90 0x55e038cfbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 -- 172.21.15.7:0/1379217614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5a90 msgr2=0x55e038cfbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5a90 0x55e038cfbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 -- 172.21.15.7:0/1379217614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cfe670 msgr2=0x55e038cf13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cfe670 0x55e038cf13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 -- 172.21.15.7:0/1379217614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e038cf58c0 con 0x55e038cf5170 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.855+0000 7f77c2800640 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77b409c7d0 tx=0x7f77b4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d0ac3aca756b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.856+0000 7f77b2a00640 1 -- 172.21.15.7:0/1379217614 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b409ab70 con 0x55e038cf5170 2024-04-20T21:43:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.856+0000 7f77b2a00640 1 -- 172.21.15.7:0/1379217614 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e038c67a70 con 0x55e038cf5170 2024-04-20T21:43:11.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.856+0000 7f77b2a00640 1 -- 172.21.15.7:0/1379217614 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4018b30 con 0x55e038cf5170 2024-04-20T21:43:11.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 -- 172.21.15.7:0/1379217614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 msgr2=0x55e038d0a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f77b409c7d0 tx=0x7f77b4018420 comp rx=0 tx=0).stop 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 -- 172.21.15.7:0/1379217614 shutdown_connections 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cfe670 0x55e038cf13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5a90 0x55e038cfbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/1379217614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cf5170 0x55e038d0a350 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 -- 172.21.15.7:0/1379217614 >> 172.21.15.7:0/1379217614 conn(0x55e038cef8a0 msgr2=0x55e038cf0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.857+0000 7f77cc1c2740 1 -- 172.21.15.7:0/1379217614 shutdown_connections 2024-04-20T21:43:11.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.858+0000 7f77cc1c2740 1 -- 172.21.15.7:0/1379217614 wait complete. 2024-04-20T21:43:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.860+0000 7f77cc1c2740 1 -- start start 2024-04-20T21:43:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.860+0000 7f77cc1c2740 1 Processor -- start 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038d022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038d02830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b2830 con 0x55e038cf5a90 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b3c80 con 0x55e038cfe670 2024-04-20T21:43:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41b2ee0 con 0x55e038cf5170 2024-04-20T21:43:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.862+0000 7f77cc1c2740 10 client.0 _ll_register_callbacks cb 0x7f77ac0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:40172/0 (socket says 172.21.15.7:40172) 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 -- 172.21.15.7:0/3920249356 learned_addr learned my addr 172.21.15.7:0/3920249356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c2800640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038d022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77bbe00640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038d02830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 -- 172.21.15.7:0/3920249356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 msgr2=0x55e038d022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038d022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 -- 172.21.15.7:0/3920249356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 msgr2=0x55e038d02830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038d02830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.863+0000 7f77c3200640 1 -- 172.21.15.7:0/3920249356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e03896c3f0 con 0x55e038cfe670 2024-04-20T21:43:11.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.864+0000 7f77c3200640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0124c0 tx=0x7f77bc0124f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d1490e08e7415 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:11.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.864+0000 7f77a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:11.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.866+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc005100 con 0x55e038cfe670 2024-04-20T21:43:11.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.866+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f77bc0052a0 con 0x55e038cfe670 2024-04-20T21:43:11.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.866+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e038c67a70 con 0x55e038cfe670 2024-04-20T21:43:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.868+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f77bc0054a0 con 0x55e038cfe670 2024-04-20T21:43:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.869+0000 7f77cc1c2740 10 client.27134 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.869+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e038a367d0 con 0x55e038cfe670 2024-04-20T21:43:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e038c67a70 con 0x55e038cfe670 2024-04-20T21:43:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f77bc02f070 con 0x55e038cfe670 2024-04-20T21:43:11.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 10 client.27134 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:11.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 10 client.27134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:11.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 20 client.27134 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:11.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d750f0 0x55e038d77510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:11.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 4 client.27134 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:11.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e038c618e0 con 0x55e038d750f0 2024-04-20T21:43:11.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.870+0000 7f77c2800640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d750f0 0x55e038d77510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:11.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.871+0000 7f77c2800640 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d750f0 0x55e038d77510 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19dbb68e67ee0cde server_cookie=e54191c9d291fe0a in_seq=0 out_seq=0 2024-04-20T21:43:11.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.871+0000 7f77a7e00640 10 client.27134 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:11.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.872+0000 7f77a7e00640 1 -- 172.21.15.7:0/3920249356 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e038c618e0 con 0x55e038d750f0 2024-04-20T21:43:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.872+0000 7f77a7e00640 10 client.27134 handle_command_reply: tid=0 2024-04-20T21:43:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.873+0000 7f77cc1c2740 1 client.27134 shutdown 2024-04-20T21:43:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 msgr2=0x55e038d02d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0124c0 tx=0x7f77bc0124f0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 shutdown_connections 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e038cfe670 0x55e038d02d70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55e038d750f0 0x55e038d77510 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e038cf5a90 0x55e038d02830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77a7e00640 0 client.27134 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 --2- 172.21.15.7:0/3920249356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e038cf5170 0x55e038d022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.875+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 >> 172.21.15.7:0/3920249356 conn(0x55e038cef8a0 msgr2=0x55e038cf1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.878+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 shutdown_connections 2024-04-20T21:43:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.879+0000 7f77cc1c2740 1 -- 172.21.15.7:0/3920249356 wait complete. 2024-04-20T21:43:11.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.880+0000 7f77cc1c2740 20 client.27134 trim_cache size 0 max 16384 2024-04-20T21:43:11.883 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 msgr2=0x7f77a80a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 0x7f77a80a62e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f77bc005b40 tx=0x7f77bc010fd0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 msgr2=0x7f77c41a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f77b400e9f0 tx=0x7f77b400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 shutdown_connections 2024-04-20T21:43:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c419a710 0x7f77c419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77c4105f90 0x7f77c41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f77a80a3e20 0x7f77a80a62e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 --2- 172.21.15.7:0/82442830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c4100ec0 0x7f77c41a0650 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 >> 172.21.15.7:0/82442830 conn(0x7f77c40fc7a0 msgr2=0x7f77c410dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.886+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 shutdown_connections 2024-04-20T21:43:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:11.887+0000 7f77c8a00640 1 -- 172.21.15.7:0/82442830 wait complete. 2024-04-20T21:43:11.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:11 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2603457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:11.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:11 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2603457343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:12 smithi007 ceph-mon[16867]: pgmap v306: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:12.916 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-20T21:43:12.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:43:12.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-20T21:43:12.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:12 smithi016 ceph-mon[21409]: pgmap v306: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:12.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:12 smithi186 ceph-mon[22157]: pgmap v306: 97 pgs: 40 active+clean+snaptrim_wait, 33 active+clean, 24 active+clean+snaptrim; 16 GiB data, 35 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:43:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 -- 172.21.15.7:0/3994514746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 msgr2=0x7f43fc109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 --2- 172.21.15.7:0/3994514746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc109990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009900 tx=0x7f43e402bc60 comp rx=0 tx=0).stop 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 -- 172.21.15.7:0/3994514746 shutdown_connections 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 --2- 172.21.15.7:0/3994514746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc114c00 0x7f43fc117010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 --2- 172.21.15.7:0/3994514746 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc109990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 --2- 172.21.15.7:0/3994514746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc06e240 0x7f43fc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 -- 172.21.15.7:0/3994514746 >> 172.21.15.7:0/3994514746 conn(0x7f43fc06ce70 msgr2=0x7f43fc06d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 -- 172.21.15.7:0/3994514746 shutdown_connections 2024-04-20T21:43:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.253+0000 7f4402200640 1 -- 172.21.15.7:0/3994514746 wait complete. 2024-04-20T21:43:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.255+0000 7f4402200640 1 Processor -- start 2024-04-20T21:43:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.255+0000 7f4402200640 1 -- start start 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 0x7f43fc1a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc1a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 0x7f43fc1a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc11a960 con 0x7f43fc06e240 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc11a7e0 con 0x7f43fc114c00 2024-04-20T21:43:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4402200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc11aae0 con 0x7f43fc109530 2024-04-20T21:43:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc1a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc1a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50356/0 (socket says 172.21.15.7:50356) 2024-04-20T21:43:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f4401800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 0x7f43fc1a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.256+0000 7f43f3e00640 1 -- 172.21.15.7:0/1329933916 learned_addr learned my addr 172.21.15.7:0/1329933916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.257+0000 7f4401800640 1 -- 172.21.15.7:0/1329933916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 msgr2=0x7f43fc1a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.257+0000 7f4401800640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc1a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.257+0000 7f4401800640 1 -- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 msgr2=0x7f43fc1a30c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:43:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.257+0000 7f4400e00640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 0x7f43fc1a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.260+0000 7f4401800640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 0x7f43fc1a30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.260+0000 7f4401800640 1 -- 172.21.15.7:0/1329933916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc1a80b0 con 0x7f43fc114c00 2024-04-20T21:43:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.260+0000 7f4400e00640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 0x7f43fc1a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.260+0000 7f4401800640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 0x7f43fc1a7980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e940 tx=0x7f43f400ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.262+0000 7f43fa000640 1 -- 172.21.15.7:0/1329933916 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f400ccc0 con 0x7f43fc114c00 2024-04-20T21:43:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.262+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc1a83a0 con 0x7f43fc114c00 2024-04-20T21:43:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.263+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1afc30 con 0x7f43fc114c00 2024-04-20T21:43:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.266+0000 7f43fa000640 1 -- 172.21.15.7:0/1329933916 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f4005020 con 0x7f43fc114c00 2024-04-20T21:43:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.266+0000 7f43fa000640 1 -- 172.21.15.7:0/1329933916 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f401c070 con 0x7f43fc114c00 2024-04-20T21:43:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.268+0000 7f4406611740 1 Processor -- start 2024-04-20T21:43:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.269+0000 7f43fa000640 1 -- 172.21.15.7:0/1329933916 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f43f4004a60 con 0x7f43fc114c00 2024-04-20T21:43:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.269+0000 7f4406611740 1 -- start start 2024-04-20T21:43:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5614d8a51d20 con 0x5614d8af65f0 2024-04-20T21:43:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614d89f83e0 con 0x5614d8aed0f0 2024-04-20T21:43:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.270+0000 7f4406611740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5614d8a4dce0 con 0x5614d8aeda10 2024-04-20T21:43:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.271+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.271+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57568/0 (socket says 172.21.15.7:57568) 2024-04-20T21:43:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.271+0000 7f43f3e00640 1 -- 172.21.15.7:0/1088590949 learned_addr learned my addr 172.21.15.7:0/1088590949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.271+0000 7f43f3e00640 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43f3e00640 1 -- 172.21.15.7:0/1088590949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 msgr2=0x5614d8af60b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43f3e00640 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43f3e00640 1 -- 172.21.15.7:0/1088590949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 msgr2=0x5614d8af89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43f3e00640 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43f3e00640 1 -- 172.21.15.7:0/1088590949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d8a17980 con 0x5614d8aed0f0 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43fa000640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 0x7f43e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.272+0000 7f43fa000640 1 -- 172.21.15.7:0/1329933916 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f43f40d04c0 con 0x7f43fc114c00 2024-04-20T21:43:13.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.273+0000 7f43f3e00640 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005ec0 tx=0x7f43e4005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f341cb00ff9e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.273+0000 7f4400e00640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 0x7f43e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.273+0000 7f43f2000640 1 -- 172.21.15.7:0/1088590949 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4047070 con 0x5614d8aed0f0 2024-04-20T21:43:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.274+0000 7f43f2000640 1 -- 172.21.15.7:0/1088590949 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e402cae0 con 0x5614d8aed0f0 2024-04-20T21:43:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.275+0000 7f43f2000640 1 -- 172.21.15.7:0/1088590949 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e40052f0 con 0x5614d8aed0f0 2024-04-20T21:43:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.275+0000 7f4406611740 1 -- 172.21.15.7:0/1088590949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 msgr2=0x5614d8aed4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.275+0000 7f4406611740 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005ec0 tx=0x7f43e4005e30 comp rx=0 tx=0).stop 2024-04-20T21:43:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.276+0000 7f4406611740 1 -- 172.21.15.7:0/1088590949 shutdown_connections 2024-04-20T21:43:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.276+0000 7f4406611740 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.276+0000 7f4406611740 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.276+0000 7f4406611740 1 --2- 172.21.15.7:0/1088590949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8aed4d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.276+0000 7f4406611740 1 -- 172.21.15.7:0/1088590949 >> 172.21.15.7:0/1088590949 conn(0x5614d8ae7820 msgr2=0x5614d8ae7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.277+0000 7f4406611740 1 -- 172.21.15.7:0/1088590949 shutdown_connections 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.277+0000 7f4406611740 1 -- 172.21.15.7:0/1088590949 wait complete. 2024-04-20T21:43:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.277+0000 7f4400e00640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 0x7f43e00a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5614d8aecbc0 tx=0x7f43ec0093c0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.278+0000 7f4406611740 1 -- start start 2024-04-20T21:43:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.278+0000 7f4406611740 1 Processor -- start 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8af3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5614d8af9420 con 0x5614d8af65f0 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5614d8a0b7a0 con 0x5614d8aed0f0 2024-04-20T21:43:13.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x5614d8af95a0 con 0x5614d8aeda10 2024-04-20T21:43:13.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4406611740 10 client.0 _ll_register_callbacks cb 0x5614d8a0db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:13.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50376/0 (socket says 172.21.15.7:50376) 2024-04-20T21:43:13.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f43f3e00640 1 -- 172.21.15.7:0/2628756328 learned_addr learned my addr 172.21.15.7:0/2628756328 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.281+0000 7f4400e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8af3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.282+0000 7f43f3e00640 1 -- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 msgr2=0x5614d8af3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.282+0000 7f43f3e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8af3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.282+0000 7f43f3e00640 1 -- 172.21.15.7:0/2628756328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 msgr2=0x5614d8af3f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.283+0000 7f4401800640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.283+0000 7f43f3e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.283+0000 7f43f3e00640 1 -- 172.21.15.7:0/2628756328 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d882c7e0 con 0x5614d8aeda10 2024-04-20T21:43:13.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.283+0000 7f4401800640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:13.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.284+0000 7f43f3e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005a90 tx=0x7f43e4004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3b3a2babe4cc63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.284+0000 7f43eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:13.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.284+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e404f060 con 0x5614d8aeda10 2024-04-20T21:43:13.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.284+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f43e402c8d0 con 0x5614d8aeda10 2024-04-20T21:43:13.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.286+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43e402cae0 con 0x5614d8aeda10 2024-04-20T21:43:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.288+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43e4047070 con 0x5614d8aeda10 2024-04-20T21:43:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.288+0000 7f4406611740 10 client.27088 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.288+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614d888cf70 con 0x5614d8aeda10 2024-04-20T21:43:13.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.288+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e402cae0 con 0x5614d8aeda10 2024-04-20T21:43:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.291+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f43e403c030 con 0x5614d8aeda10 2024-04-20T21:43:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.291+0000 7f4406611740 10 client.27088 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.291+0000 7f4406611740 10 client.27088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.291+0000 7f4406611740 20 client.27088 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.291+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b06810 0x5614d8b08c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.292+0000 7f4406611740 4 client.27088 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.292+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614d8a5f740 con 0x5614d8b06810 2024-04-20T21:43:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.296+0000 7f4400e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b06810 0x5614d8b08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.298+0000 7f4400e00640 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b06810 0x5614d8b08c00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0bd73f36932a3b server_cookie=aabed3218135e884 in_seq=0 out_seq=0 2024-04-20T21:43:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.298+0000 7f43eaa00640 10 client.27088 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.299+0000 7f43eaa00640 1 -- 172.21.15.7:0/2628756328 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614d8a5f740 con 0x5614d8b06810 2024-04-20T21:43:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.299+0000 7f43eaa00640 10 client.27088 handle_command_reply: tid=0 2024-04-20T21:43:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.299+0000 7f4406611740 1 client.27088 shutdown 2024-04-20T21:43:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.300+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 msgr2=0x5614d8af3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005a90 tx=0x7f43e4004ef0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 shutdown_connections 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b06810 0x5614d8b08c00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8af3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8af3a40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 --2- 172.21.15.7:0/2628756328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aed0f0 0x5614d8af3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 >> 172.21.15.7:0/2628756328 conn(0x5614d8ae7820 msgr2=0x5614d8ae9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.301+0000 7f43eaa00640 0 client.27088 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.302+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 shutdown_connections 2024-04-20T21:43:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.302+0000 7f4406611740 1 -- 172.21.15.7:0/2628756328 wait complete. 2024-04-20T21:43:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.302+0000 7f4406611740 20 client.27088 trim_cache size 0 max 16384 2024-04-20T21:43:13.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.346+0000 7f4406611740 1 Processor -- start 2024-04-20T21:43:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.346+0000 7f4406611740 1 -- start start 2024-04-20T21:43:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8aed0f0 0x5614d8ae6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8b0b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8011010 con 0x5614d8aed0f0 2024-04-20T21:43:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800e8d0 con 0x5614d8af65f0 2024-04-20T21:43:13.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.347+0000 7f4406611740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800c720 con 0x5614d8aeda10 2024-04-20T21:43:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8b0b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8b0b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50386/0 (socket says 172.21.15.7:50386) 2024-04-20T21:43:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f4401800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f4401800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57588/0 (socket says 172.21.15.7:57588) 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f4401800640 1 -- 172.21.15.7:0/265017921 learned_addr learned my addr 172.21.15.7:0/265017921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.348+0000 7f4400e00640 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8aed0f0 0x5614d8ae6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 -- 172.21.15.7:0/265017921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 msgr2=0x5614d8b0b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8b0b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 -- 172.21.15.7:0/265017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8aed0f0 msgr2=0x5614d8ae6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8aed0f0 0x5614d8ae6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 -- 172.21.15.7:0/265017921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614d8724170 con 0x5614d8af65f0 2024-04-20T21:43:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.349+0000 7f4401800640 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f43f4099e60 tx=0x7f43f409db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5cd568920acd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.350+0000 7f43f1600640 1 -- 172.21.15.7:0/265017921 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f40b4020 con 0x5614d8af65f0 2024-04-20T21:43:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.350+0000 7f43f1600640 1 -- 172.21.15.7:0/265017921 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f4042070 con 0x5614d8af65f0 2024-04-20T21:43:13.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.350+0000 7f43f1600640 1 -- 172.21.15.7:0/265017921 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f409c8e0 con 0x5614d8af65f0 2024-04-20T21:43:13.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.351+0000 7f4406611740 1 -- 172.21.15.7:0/265017921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 msgr2=0x5614d8b0bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.351+0000 7f4406611740 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f43f4099e60 tx=0x7f43f409db90 comp rx=0 tx=0).stop 2024-04-20T21:43:13.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.351+0000 7f4406611740 1 -- 172.21.15.7:0/265017921 shutdown_connections 2024-04-20T21:43:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8af65f0 0x5614d8b0bd80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aeda10 0x5614d8b0b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 --2- 172.21.15.7:0/265017921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8aed0f0 0x5614d8ae6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 -- 172.21.15.7:0/265017921 >> 172.21.15.7:0/265017921 conn(0x5614d8ae7820 msgr2=0x5614d8ae6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 -- 172.21.15.7:0/265017921 shutdown_connections 2024-04-20T21:43:13.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.352+0000 7f4406611740 1 -- 172.21.15.7:0/265017921 wait complete. 2024-04-20T21:43:13.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.353+0000 7f4406611740 1 -- start start 2024-04-20T21:43:13.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.353+0000 7f4406611740 1 Processor -- start 2024-04-20T21:43:13.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aed0f0 0x5614d8b093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8b09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800cd90 con 0x5614d8af65f0 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800e200 con 0x5614d8aeda10 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f43d800d460 con 0x5614d8aed0f0 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f4406611740 10 client.0 _ll_register_callbacks cb 0x7f43e4045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57592/0 (socket says 172.21.15.7:57592) 2024-04-20T21:43:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.354+0000 7f43f3e00640 1 -- 172.21.15.7:0/3703939803 learned_addr learned my addr 172.21.15.7:0/3703939803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f43f3e00640 1 -- 172.21.15.7:0/3703939803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aed0f0 msgr2=0x5614d8b093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f43f3e00640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aed0f0 0x5614d8b093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f4401800640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8b09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f43f3e00640 1 -- 172.21.15.7:0/3703939803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 msgr2=0x5614d8b09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f43f3e00640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8b09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f43f3e00640 1 -- 172.21.15.7:0/3703939803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614d8b6bbf0 con 0x5614d8aeda10 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.355+0000 7f4401800640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8b09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:43:13.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.356+0000 7f43f3e00640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006010 tx=0x7f43e40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95effd28a5ed4c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.356+0000 7f43eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:13.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.359+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4044020 con 0x5614d8aeda10 2024-04-20T21:43:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.359+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f43e4039040 con 0x5614d8aeda10 2024-04-20T21:43:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.359+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f4042070 con 0x5614d8aeda10 2024-04-20T21:43:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.360+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43e4062050 con 0x5614d8aeda10 2024-04-20T21:43:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.362+0000 7f4406611740 10 client.27149 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.362+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614d8b6cef0 con 0x5614d8aeda10 2024-04-20T21:43:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.363+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f4042070 con 0x5614d8aeda10 2024-04-20T21:43:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.363+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f43e403c050 con 0x5614d8aeda10 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 10 client.27149 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 20 client.27149 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b6d5a0 0x5614d8b6f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 4 client.27149 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5614d8a595b0 con 0x5614d8b6d5a0 2024-04-20T21:43:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.365+0000 7f4400e00640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b6d5a0 0x5614d8b6f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.367+0000 7f4400e00640 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b6d5a0 0x5614d8b6f990 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989d0613f78265de server_cookie=f270ce82ce4c85b0 in_seq=0 out_seq=0 2024-04-20T21:43:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.367+0000 7f43eaa00640 10 client.27149 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.372+0000 7f43eaa00640 1 -- 172.21.15.7:0/3703939803 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5614d8a595b0 con 0x5614d8b6d5a0 2024-04-20T21:43:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.372+0000 7f43eaa00640 10 client.27149 handle_command_reply: tid=0 2024-04-20T21:43:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.372+0000 7f4406611740 1 client.27149 shutdown 2024-04-20T21:43:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.372+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 msgr2=0x5614d8b09930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.372+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006010 tx=0x7f43e40046c0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 shutdown_connections 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5614d8aeda10 0x5614d8b09930 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f43e4006010 tx=0x7f43e40046c0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x5614d8b6d5a0 0x5614d8b6f990 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614d8af65f0 0x5614d8b09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 --2- 172.21.15.7:0/3703939803 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5614d8aed0f0 0x5614d8b093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 >> 172.21.15.7:0/3703939803 conn(0x5614d8ae7820 msgr2=0x5614d8b02160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.373+0000 7f43eaa00640 0 client.27149 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.374+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 shutdown_connections 2024-04-20T21:43:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.374+0000 7f4406611740 1 -- 172.21.15.7:0/3703939803 wait complete. 2024-04-20T21:43:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.374+0000 7f4406611740 20 client.27149 trim_cache size 0 max 16384 2024-04-20T21:43:13.375 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-04-20T21:43:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "3e07f806-40aa-4ea3-bbca-f79fc8482dce", 2024-04-20T21:43:13.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-04-20T21:43:13.375 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 msgr2=0x7f43e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 0x7f43e00a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5614d8aecbc0 tx=0x7f43ec0093c0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 msgr2=0x7f43fc1a7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 0x7f43fc1a7980 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f43f400e940 tx=0x7f43f400ee00 comp rx=0 tx=0).stop 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 shutdown_connections 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43e00a3da0 0x7f43e00a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43fc114c00 0x7f43fc1a7980 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f43fc109530 0x7f43fc1a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 --2- 172.21.15.7:0/1329933916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43fc06e240 0x7f43fc1a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.375+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 >> 172.21.15.7:0/1329933916 conn(0x7f43fc06ce70 msgr2=0x7f43fc10a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.376+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 shutdown_connections 2024-04-20T21:43:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.376+0000 7f4402200640 1 -- 172.21.15.7:0/1329933916 wait complete. 2024-04-20T21:43:13.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:13.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.743+0000 7f5ebe200640 1 -- 172.21.15.7:0/779216404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb8109b20 msgr2=0x7f5eb810e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.743+0000 7f5ebe200640 1 --2- 172.21.15.7:0/779216404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb8109b20 0x7f5eb810e7b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8009860 tx=0x7f5ea802bb70 comp rx=0 tx=0).stop 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.743+0000 7f5ebe200640 1 -- 172.21.15.7:0/779216404 shutdown_connections 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.743+0000 7f5ebe200640 1 --2- 172.21.15.7:0/779216404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81133e0 0x7f5eb81157d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.744+0000 7f5ebe200640 1 --2- 172.21.15.7:0/779216404 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb8109b20 0x7f5eb810e7b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.744+0000 7f5ebe200640 1 --2- 172.21.15.7:0/779216404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb81096b0 0x7f5eb810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.744+0000 7f5ebe200640 1 -- 172.21.15.7:0/779216404 >> 172.21.15.7:0/779216404 conn(0x7f5eb806d000 msgr2=0x7f5eb806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.752+0000 7f5ebe200640 1 -- 172.21.15.7:0/779216404 shutdown_connections 2024-04-20T21:43:13.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.753+0000 7f5ebe200640 1 -- 172.21.15.7:0/779216404 wait complete. 2024-04-20T21:43:13.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.755+0000 7f5ebe200640 1 Processor -- start 2024-04-20T21:43:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.755+0000 7f5ebe200640 1 -- start start 2024-04-20T21:43:13.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.755+0000 7f5ebe200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.755+0000 7f5ebe200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb8109b20 0x7f5eb81a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5ebe200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb81133e0 0x7f5eb8112070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5ebe200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb811ab90 con 0x7f5eb8109b20 2024-04-20T21:43:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5ebe200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb811aa10 con 0x7f5eb81096b0 2024-04-20T21:43:13.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5ebe200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb811ad10 con 0x7f5eb81133e0 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57608/0 (socket says 172.21.15.7:57608) 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5eb7400640 1 -- 172.21.15.7:0/3682586400 learned_addr learned my addr 172.21.15.7:0/3682586400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.756+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb81133e0 0x7f5eb8112070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.757+0000 7f5eb7400640 1 -- 172.21.15.7:0/3682586400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb81133e0 msgr2=0x7f5eb8112070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.757+0000 7f5eb7400640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb81133e0 0x7f5eb8112070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.757+0000 7f5eb7400640 1 -- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb8109b20 msgr2=0x7f5eb81a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.757+0000 7f5eb7400640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb8109b20 0x7f5eb81a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.757+0000 7f5eb7400640 1 -- 172.21.15.7:0/3682586400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb81127f0 con 0x7f5eb81096b0 2024-04-20T21:43:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.758+0000 7f5eb7400640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9c0 tx=0x7f5ea400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.759+0000 7f5eafe00640 1 -- 172.21.15.7:0/3682586400 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400ccc0 con 0x7f5eb81096b0 2024-04-20T21:43:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.759+0000 7f5eafe00640 1 -- 172.21.15.7:0/3682586400 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea4005500 con 0x7f5eb81096b0 2024-04-20T21:43:13.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.760+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb8112ae0 con 0x7f5eb81096b0 2024-04-20T21:43:13.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.760+0000 7f5eafe00640 1 -- 172.21.15.7:0/3682586400 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4010430 con 0x7f5eb81096b0 2024-04-20T21:43:13.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.760+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81afe10 con 0x7f5eb81096b0 2024-04-20T21:43:13.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.768+0000 7f5ec0f9d740 1 Processor -- start 2024-04-20T21:43:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.768+0000 7f5ec0f9d740 1 -- start start 2024-04-20T21:43:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db8d41c150 con 0x55db8d4b61a0 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db8d3c2370 con 0x55db8d4bf6a0 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55db8d418110 con 0x55db8d4b6ac0 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5eafe00640 1 -- 172.21.15.7:0/3682586400 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f5ea40046c0 con 0x7f5eb81096b0 2024-04-20T21:43:13.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50420/0 (socket says 172.21.15.7:50420) 2024-04-20T21:43:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.769+0000 7f5eb7400640 1 -- 172.21.15.7:0/744150461 learned_addr learned my addr 172.21.15.7:0/744150461 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.770+0000 7f5eb7400640 1 -- 172.21.15.7:0/744150461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 msgr2=0x55db8d4c1a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.770+0000 7f5eb7400640 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4c1a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.770+0000 7f5eb7400640 1 -- 172.21.15.7:0/744150461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 msgr2=0x55db8d4b6580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.770+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.771+0000 7f5eb7400640 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.771+0000 7f5eb7400640 1 -- 172.21.15.7:0/744150461 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8d1f87d0 con 0x55db8d4b6ac0 2024-04-20T21:43:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.771+0000 7f5eb7400640 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40cdf20 tx=0x7f5ea40d6ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff1b3282a8d4eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.772+0000 7f5ea3e00640 1 -- 172.21.15.7:0/744150461 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40df070 con 0x55db8d4b6ac0 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.772+0000 7f5ea3e00640 1 -- 172.21.15.7:0/744150461 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea40d8e10 con 0x55db8d4b6ac0 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.772+0000 7f5eafe00640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 0x7f5e9c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.773+0000 7f5eafe00640 1 -- 172.21.15.7:0/3682586400 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f5ea40d08d0 con 0x7f5eb81096b0 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.773+0000 7f5ea3e00640 1 -- 172.21.15.7:0/744150461 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40d25b0 con 0x55db8d4b6ac0 2024-04-20T21:43:13.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.774+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 0x7f5e9c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.775+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/744150461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 msgr2=0x55db8d4bf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.775+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40cdf20 tx=0x7f5ea40d6ef0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.776+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/744150461 shutdown_connections 2024-04-20T21:43:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.776+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4c1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.776+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bf160 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.776+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/744150461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.776+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/744150461 >> 172.21.15.7:0/744150461 conn(0x55db8d4b08d0 msgr2=0x55db8d4b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.777+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 0x7f5e9c0a62e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5eb806b570 tx=0x7f5eb000a480 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.777+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/744150461 shutdown_connections 2024-04-20T21:43:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.777+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/744150461 wait complete. 2024-04-20T21:43:13.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.778+0000 7f5ec0f9d740 1 -- start start 2024-04-20T21:43:13.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.778+0000 7f5ec0f9d740 1 Processor -- start 2024-04-20T21:43:13.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db8d4c2490 con 0x55db8d4b61a0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55db8d3d5740 con 0x55db8d4bf6a0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.780+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55db8d4c2610 con 0x55db8d4b6ac0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4bc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52738/0 (socket says 172.21.15.7:52738) 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7400640 1 -- 172.21.15.7:0/1098239625 learned_addr learned my addr 172.21.15.7:0/1098239625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb6a00640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5ec0f9d740 10 client.0 _ll_register_callbacks cb 0x55db8d3d0b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 -- 172.21.15.7:0/1098239625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 msgr2=0x55db8d4bcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 -- 172.21.15.7:0/1098239625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 msgr2=0x55db8d4bc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4bc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.781+0000 7f5eb7e00640 1 -- 172.21.15.7:0/1098239625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8d12e3f0 con 0x55db8d4bf6a0 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.782+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea40d8e10 con 0x55db8d4bf6a0 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.782+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4bd150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0019e10 tx=0x7f5eb0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624ad755b5d4ac48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.782+0000 7f5ea0c00640 10 client.27155 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.783+0000 7f5ea0c00640 1 -- 172.21.15.7:0/1098239625 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0027060 con 0x55db8d4bf6a0 2024-04-20T21:43:13.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.784+0000 7f5ea0c00640 1 -- 172.21.15.7:0/1098239625 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f5eb001f070 con 0x55db8d4bf6a0 2024-04-20T21:43:13.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.784+0000 7f5ea0c00640 1 -- 172.21.15.7:0/1098239625 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5eb0027060 con 0x55db8d4bf6a0 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.784+0000 7f5ec0f9d740 10 client.27155 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.784+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55db8d0ee0f0 con 0x55db8d4bf6a0 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.785+0000 7f5ea0c00640 1 -- 172.21.15.7:0/1098239625 <== mon.2 v2:172.21.15.16:3300/0 4 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f5eb002a050 con 0x55db8d4bf6a0 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.785+0000 7f5ec0f9d740 10 client.27155 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.785+0000 7f5ec0f9d740 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.785+0000 7f5ec0f9d740 20 client.27155 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.786+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4cfdf0 0x55db8d4d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.786+0000 7f5ec0f9d740 4 client.27155 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.786+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db8d429b70 con 0x55db8d4cfdf0 2024-04-20T21:43:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.787+0000 7f5eb7400640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4cfdf0 0x55db8d4d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.790+0000 7f5eb7400640 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4cfdf0 0x55db8d4d2230 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ce8e6ca29954b server_cookie=5726c743892e8401 in_seq=0 out_seq=0 2024-04-20T21:43:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.790+0000 7f5ea0c00640 10 client.27155 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.791+0000 7f5ea0c00640 1 -- 172.21.15.7:0/1098239625 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55db8d429b70 con 0x55db8d4cfdf0 2024-04-20T21:43:13.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.791+0000 7f5ea0c00640 10 client.27155 handle_command_reply: tid=0 2024-04-20T21:43:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.792+0000 7f5ec0f9d740 1 client.27155 shutdown 2024-04-20T21:43:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.793+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 msgr2=0x55db8d4bd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.793+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4bd150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0019e10 tx=0x7f5eb0004a40 comp rx=0 tx=0).stop 2024-04-20T21:43:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 shutdown_connections 2024-04-20T21:43:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4bf6a0 0x55db8d4bd150 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4cfdf0 0x55db8d4d2230 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/1098239625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b61a0 0x55db8d4bc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 >> 172.21.15.7:0/1098239625 conn(0x55db8d4b08d0 msgr2=0x55db8d4b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.794+0000 7f5ea0c00640 0 client.27155 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.795+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 shutdown_connections 2024-04-20T21:43:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.795+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/1098239625 wait complete. 2024-04-20T21:43:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.796+0000 7f5ec0f9d740 20 client.27155 trim_cache size 0 max 16384 2024-04-20T21:43:13.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.843+0000 7f5ec0f9d740 1 Processor -- start 2024-04-20T21:43:13.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.843+0000 7f5ec0f9d740 1 -- start start 2024-04-20T21:43:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b08d0 0x55db8d4bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81bb150 con 0x55db8d4b08d0 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81b8a10 con 0x55db8d4b61a0 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81b6860 con 0x55db8d4b6ac0 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.845+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b08d0 0x55db8d4bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.846+0000 7f5eb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.846+0000 7f5eb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57640/0 (socket says 172.21.15.7:57640) 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.846+0000 7f5eb6a00640 1 -- 172.21.15.7:0/347560650 learned_addr learned my addr 172.21.15.7:0/347560650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.846+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.847+0000 7f5eb6a00640 1 -- 172.21.15.7:0/347560650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 msgr2=0x55db8d4bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.847+0000 7f5eb6a00640 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.847+0000 7f5eb6a00640 1 -- 172.21.15.7:0/347560650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b08d0 msgr2=0x55db8d4bb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.847+0000 7f5eb6a00640 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b08d0 0x55db8d4bb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.847+0000 7f5eb6a00640 1 -- 172.21.15.7:0/347560650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8d4bac20 con 0x55db8d4b61a0 2024-04-20T21:43:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.848+0000 7f5eb6a00640 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8037040 tx=0x7f5ea8032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c2221f8395b990 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.848+0000 7f5ea3400640 1 -- 172.21.15.7:0/347560650 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8034ab0 con 0x55db8d4b61a0 2024-04-20T21:43:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.849+0000 7f5ea3400640 1 -- 172.21.15.7:0/347560650 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8034c50 con 0x55db8d4b61a0 2024-04-20T21:43:13.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.850+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/347560650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 msgr2=0x55db8d4cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.850+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8037040 tx=0x7f5ea8032340 comp rx=0 tx=0).stop 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/347560650 shutdown_connections 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b61a0 0x55db8d4cb380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b6ac0 0x55db8d4bcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/347560650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b08d0 0x55db8d4bb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/347560650 >> 172.21.15.7:0/347560650 conn(0x55db8d4b80a0 msgr2=0x55db8d4af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.851+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/347560650 shutdown_connections 2024-04-20T21:43:13.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.852+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/347560650 wait complete. 2024-04-20T21:43:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.852+0000 7f5ec0f9d740 1 -- start start 2024-04-20T21:43:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.853+0000 7f5ec0f9d740 1 Processor -- start 2024-04-20T21:43:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 0x55db8d4c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 0x55db8d4c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5eb7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 0x55db8d4c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:57654/0 (socket says 172.21.15.7:57654) 2024-04-20T21:43:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5ec0f9d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 0x55db8d4c3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.854+0000 7f5eb6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5eb6a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:50452/0 (socket says 172.21.15.7:50452) 2024-04-20T21:43:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81b6ed0 con 0x55db8d4b6ac0 2024-04-20T21:43:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81b8340 con 0x55db8d4b08d0 2024-04-20T21:43:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5ec0f9d740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb81b75a0 con 0x55db8d4b61a0 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5ec0f9d740 10 client.0 _ll_register_callbacks cb 0x7f5eb000ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5eb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 0x55db8d4c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.855+0000 7f5eb7400640 1 -- 172.21.15.7:0/2941004530 learned_addr learned my addr 172.21.15.7:0/2941004530 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb7e00640 1 -- 172.21.15.7:0/2941004530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 msgr2=0x55db8d4c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb7e00640 1 -- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 msgr2=0x55db8d4c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 0x55db8d4c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb7e00640 1 -- 172.21.15.7:0/2941004530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8d4c3e00 con 0x55db8d4b6ac0 2024-04-20T21:43:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.856+0000 7f5eb6a00640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-20T21:43:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.857+0000 7f5eb7e00640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 0x55db8d4c3580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0019860 tx=0x7f5eb001b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b7670f8ec3088e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.857+0000 7f5ea0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:43:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.858+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0021aa0 con 0x55db8d4b6ac0 2024-04-20T21:43:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.858+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f5eb0021c40 con 0x55db8d4b6ac0 2024-04-20T21:43:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.858+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ea8034c50 con 0x55db8d4b6ac0 2024-04-20T21:43:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.859+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5eb0021e40 con 0x55db8d4b6ac0 2024-04-20T21:43:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.860+0000 7f5ec0f9d740 10 client.16923 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.860+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55db8d536430 con 0x55db8d4b6ac0 2024-04-20T21:43:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.861+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea8034c50 con 0x55db8d4b6ac0 2024-04-20T21:43:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.861+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f5eb002a040 con 0x55db8d4b6ac0 2024-04-20T21:43:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 10 client.16923 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 20 client.16923 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4bf6a0 0x55db8d538be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 4 client.16923 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.862+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55db8d4239e0 con 0x55db8d4bf6a0 2024-04-20T21:43:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.863+0000 7f5eb7400640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4bf6a0 0x55db8d538be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.865+0000 7f5ea0c00640 10 client.16923 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.865+0000 7f5eb7400640 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4bf6a0 0x55db8d538be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb724bf74becbb92 server_cookie=324a407d887793a9 in_seq=0 out_seq=0 2024-04-20T21:43:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ea0c00640 1 -- 172.21.15.7:0/2941004530 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55db8d4239e0 con 0x55db8d4bf6a0 2024-04-20T21:43:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ea0c00640 10 client.16923 handle_command_reply: tid=0 2024-04-20T21:43:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 client.16923 shutdown 2024-04-20T21:43:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 msgr2=0x55db8d4c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 0x55db8d4c3580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0019860 tx=0x7f5eb001b050 comp rx=0 tx=0).stop 2024-04-20T21:43:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 shutdown_connections 2024-04-20T21:43:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55db8d4bf6a0 0x55db8d538be0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8d4b6ac0 0x55db8d4c3580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55db8d4b61a0 0x55db8d4c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 --2- 172.21.15.7:0/2941004530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55db8d4b08d0 0x55db8d4c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.866+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 >> 172.21.15.7:0/2941004530 conn(0x55db8d4b80a0 msgr2=0x55db8d4b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.867+0000 7f5ea0c00640 0 client.16923 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.867+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 shutdown_connections 2024-04-20T21:43:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.867+0000 7f5ec0f9d740 1 -- 172.21.15.7:0/2941004530 wait complete. 2024-04-20T21:43:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.868+0000 7f5ec0f9d740 20 client.16923 trim_cache size 0 max 16384 2024-04-20T21:43:13.868 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "scrub active (22 inodes in the stack)", 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "3e07f806-40aa-4ea3-bbca-f79fc8482dce": { 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "3e07f806-40aa-4ea3-bbca-f79fc8482dce", 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,force" 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:13.869 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.869+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 msgr2=0x7f5e9c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.869+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 0x7f5e9c0a62e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5eb806b570 tx=0x7f5eb000a480 comp rx=0 tx=0).stop 2024-04-20T21:43:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.869+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 msgr2=0x7f5eb81a8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.869+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9c0 tx=0x7f5ea400ee80 comp rx=0 tx=0).stop 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 shutdown_connections 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5eb81133e0 0x7f5eb8112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb8109b20 0x7f5eb81a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f5e9c0a3e20 0x7f5e9c0a62e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 --2- 172.21.15.7:0/3682586400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5eb81096b0 0x7f5eb81a8f40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 >> 172.21.15.7:0/3682586400 conn(0x7f5eb806d000 msgr2=0x7f5eb81176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 shutdown_connections 2024-04-20T21:43:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:13.870+0000 7f5ebe200640 1 -- 172.21.15.7:0/3682586400 wait complete. 2024-04-20T21:43:13.900 INFO:tasks.cephfs.filesystem:scrub status for tag:3e07f806-40aa-4ea3-bbca-f79fc8482dce - {'path': '/', 'tag': '3e07f806-40aa-4ea3-bbca-f79fc8482dce', 'options': 'recursive,force'} 2024-04-20T21:43:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: pgmap v307: 97 pgs: 36 active+clean+snaptrim_wait, 43 active+clean, 18 active+clean+snaptrim; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub queued for path: / 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:43:14.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: active paths [/] 2024-04-20T21:43:14.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:14 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: pgmap v307: 97 pgs: 36 active+clean+snaptrim_wait, 43 active+clean, 18 active+clean+snaptrim; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub queued for path: / 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:43:14.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: active paths [/] 2024-04-20T21:43:14.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:14 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: pgmap v307: 97 pgs: 36 active+clean+snaptrim_wait, 43 active+clean, 18 active+clean+snaptrim; 15 GiB data, 34 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-20T21:43:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub queued for path: / 2024-04-20T21:43:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: active paths [/] 2024-04-20T21:43:14.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:14 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:15.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:15 smithi007 ceph-mon[16867]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:15.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:15 smithi016 ceph-mon[21409]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:15.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:15 smithi186 ceph-mon[22157]: scrub summary: idle+waiting paths [/] 2024-04-20T21:43:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:16 smithi007 ceph-mon[16867]: pgmap v308: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:16.845 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:16 smithi016 ceph-mon[21409]: pgmap v308: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:16 smithi186 ceph-mon[22157]: pgmap v308: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:18.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:18 smithi007 ceph-mon[16867]: pgmap v309: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:18.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:18 smithi016 ceph-mon[21409]: pgmap v309: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:18 smithi186 ceph-mon[22157]: pgmap v309: 97 pgs: 36 active+clean+snaptrim_wait, 45 active+clean, 16 active+clean+snaptrim; 14 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2024-04-20T21:43:20.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:20 smithi007 ceph-mon[16867]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-20T21:43:20.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:20 smithi016 ceph-mon[21409]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-20T21:43:20.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:20 smithi186 ceph-mon[22157]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-20T21:43:22.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:22 smithi007 ceph-mon[16867]: pgmap v311: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-20T21:43:22.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:22 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:43:22.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:22 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:43:22.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:22 smithi007 ceph-mon[16867]: scrub summary: idle 2024-04-20T21:43:22.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:22 smithi016 ceph-mon[21409]: pgmap v311: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-20T21:43:22.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:22 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:43:22.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:22 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:43:22.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:22 smithi016 ceph-mon[21409]: scrub summary: idle 2024-04-20T21:43:22.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:22 smithi186 ceph-mon[22157]: pgmap v311: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.1 KiB/s wr, 6 op/s 2024-04-20T21:43:22.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:22 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:43:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:22 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:43:22.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:22 smithi186 ceph-mon[22157]: scrub summary: idle 2024-04-20T21:43:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:24 smithi007 ceph-mon[16867]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-20T21:43:24.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:24 smithi007 ceph-mon[16867]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:43:24.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:24 smithi016 ceph-mon[21409]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-20T21:43:24.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:24 smithi016 ceph-mon[21409]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:43:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:24 smithi186 ceph-mon[22157]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-20T21:43:24.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:24 smithi186 ceph-mon[22157]: osdmap e97: 12 total, 12 up, 12 in 2024-04-20T21:43:26.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:26 smithi016 ceph-mon[21409]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:26.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:26 smithi186 ceph-mon[22157]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:27.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:26 smithi007 ceph-mon[16867]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:28.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:28 smithi016 ceph-mon[21409]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:28.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:28 smithi186 ceph-mon[22157]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:29.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:28 smithi007 ceph-mon[16867]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.0 KiB/s wr, 7 op/s 2024-04-20T21:43:30.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:30 smithi016 ceph-mon[21409]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:30.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:30 smithi186 ceph-mon[22157]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:31.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:30 smithi007 ceph-mon[16867]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:32.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-20T21:43:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 -- 172.21.15.7:0/701155701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d4410e670 msgr2=0x7f8d4410ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 --2- 172.21.15.7:0/701155701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d4410e670 0x7f8d4410ea50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009930 tx=0x7f8d3402bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 -- 172.21.15.7:0/701155701 shutdown_connections 2024-04-20T21:43:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 --2- 172.21.15.7:0/701155701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d44115430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 --2- 172.21.15.7:0/701155701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d441085b0 0x7f8d44108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:32.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 --2- 172.21.15.7:0/701155701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d4410e670 0x7f8d4410ea50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 -- 172.21.15.7:0/701155701 >> 172.21.15.7:0/701155701 conn(0x7f8d44076470 msgr2=0x7f8d44078890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 -- 172.21.15.7:0/701155701 shutdown_connections 2024-04-20T21:43:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.743+0000 7f8d49400640 1 -- 172.21.15.7:0/701155701 wait complete. 2024-04-20T21:43:32.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.744+0000 7f8d49400640 1 Processor -- start 2024-04-20T21:43:32.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.744+0000 7f8d49400640 1 -- start start 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d4419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 0x7f8d441a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d441189c0 con 0x7f8d44108f70 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44118840 con 0x7f8d441085b0 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d49400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8d44118b40 con 0x7f8d4410e670 2024-04-20T21:43:32.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d42800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d42800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:53000/0 (socket says 172.21.15.7:53000) 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d42800640 1 -- 172.21.15.7:0/1333500402 learned_addr learned my addr 172.21.15.7:0/1333500402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d39e00640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d4419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.745+0000 7f8d43200640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 0x7f8d441a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 -- 172.21.15.7:0/1333500402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 msgr2=0x7f8d441a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 0x7f8d441a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 -- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 msgr2=0x7f8d4419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d4419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 -- 172.21.15.7:0/1333500402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d441a3e10 con 0x7f8d441085b0 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d39e00640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d4419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d43200640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 0x7f8d441a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d42800640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d34004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34043020 con 0x7f8d441085b0 2024-04-20T21:43:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d340054e0 con 0x7f8d441085b0 2024-04-20T21:43:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003bb0 con 0x7f8d441085b0 2024-04-20T21:43:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.746+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d441a40a0 con 0x7f8d441085b0 2024-04-20T21:43:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.747+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d441ab990 con 0x7f8d441085b0 2024-04-20T21:43:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.748+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d44050540 con 0x7f8d441085b0 2024-04-20T21:43:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.750+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8d34046020 con 0x7f8d441085b0 2024-04-20T21:43:32.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.751+0000 7f8d3be00640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 0x7f8d240a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:32.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.752+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f8d340f2830 con 0x7f8d441085b0 2024-04-20T21:43:32.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.752+0000 7f8d39e00640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 0x7f8d240a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:32.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.752+0000 7f8d39e00640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 0x7f8d240a6250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d4410dca0 tx=0x7f8d2c005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:32.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.756+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f8d3404d020 con 0x7f8d441085b0 2024-04-20T21:43:32.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:32 smithi007 ceph-mon[16867]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:32 smithi016 ceph-mon[21409]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:32.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:32 smithi186 ceph-mon[22157]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:32.990+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8d44110ad0 con 0x7f8d240a3d90 2024-04-20T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.008+0000 7f8d3be00640 1 -- 172.21.15.7:0/1333500402 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f8d44110ad0 con 0x7f8d240a3d90 2024-04-20T21:43:33.007 INFO:teuthology.orchestra.run.smithi007.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-20T00:00:00", "created": "2024-04-20T21:33:30", "first": "2024-04-20T21:34:00", "last": "2024-04-20T21:43:00", "last_pruned": "2024-04-20T21:43:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.011+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 msgr2=0x7f8d240a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.011+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 0x7f8d240a6250 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8d4410dca0 tx=0x7f8d2c005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 msgr2=0x7f8d4419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d34004360 comp rx=0 tx=0).stop 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 shutdown_connections 2024-04-20T21:43:33.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d4410e670 0x7f8d441a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d44108f70 0x7f8d4419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8d240a3d90 0x7f8d240a6250 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 --2- 172.21.15.7:0/1333500402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d441085b0 0x7f8d4419ee20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 >> 172.21.15.7:0/1333500402 conn(0x7f8d44076470 msgr2=0x7f8d44077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 shutdown_connections 2024-04-20T21:43:33.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.012+0000 7f8d49400640 1 -- 172.21.15.7:0/1333500402 wait complete. 2024-04-20T21:43:33.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-20T21:43:33.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 -- 172.21.15.7:0/2244676286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c1008a0 msgr2=0x7ffb7c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 --2- 172.21.15.7:0/2244676286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c10ec30 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009c10 tx=0x7ffb7402bef0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 -- 172.21.15.7:0/2244676286 shutdown_connections 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 --2- 172.21.15.7:0/2244676286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c0ffee0 0x7ffb7c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 --2- 172.21.15.7:0/2244676286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c10ec30 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 --2- 172.21.15.7:0/2244676286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 0x7ffb7c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 -- 172.21.15.7:0/2244676286 >> 172.21.15.7:0/2244676286 conn(0x7ffb7c076ea0 msgr2=0x7ffb7c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 -- 172.21.15.7:0/2244676286 shutdown_connections 2024-04-20T21:43:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.239+0000 7ffb80e00640 1 -- 172.21.15.7:0/2244676286 wait complete. 2024-04-20T21:43:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.240+0000 7ffb80e00640 1 Processor -- start 2024-04-20T21:43:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.240+0000 7ffb80e00640 1 -- start start 2024-04-20T21:43:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c0ffee0 0x7ffb7c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 0x7ffb7c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c112120 con 0x7ffb7c107e80 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c111fa0 con 0x7ffb7c1008a0 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb80e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c1122a0 con 0x7ffb7c0ffee0 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:36154/0 (socket says 172.21.15.7:36154) 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb73e00640 1 -- 172.21.15.7:0/3674975428 learned_addr learned my addr 172.21.15.7:0/3674975428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb7b400640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c0ffee0 0x7ffb7c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.241+0000 7ffb7be00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 0x7ffb7c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 -- 172.21.15.7:0/3674975428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c0ffee0 msgr2=0x7ffb7c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c0ffee0 0x7ffb7c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 -- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 msgr2=0x7ffb7c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 0x7ffb7c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 -- 172.21.15.7:0/3674975428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a6a80 con 0x7ffb7c1008a0 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb73e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9f0 tx=0x7ffb6400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400ccc0 con 0x7ffb7c1008a0 2024-04-20T21:43:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a6d70 con 0x7ffb7c1008a0 2024-04-20T21:43:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6400ce60 con 0x7ffb7c1008a0 2024-04-20T21:43:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64010690 con 0x7ffb7c1008a0 2024-04-20T21:43:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.242+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a7210 con 0x7ffb7c1008a0 2024-04-20T21:43:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.243+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c102e80 con 0x7ffb7c1008a0 2024-04-20T21:43:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.246+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ffb640046c0 con 0x7ffb7c1008a0 2024-04-20T21:43:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.247+0000 7ffb78c00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 0x7ffb600a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.247+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7ffb640d0de0 con 0x7ffb7c1008a0 2024-04-20T21:43:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.247+0000 7ffb7b400640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 0x7ffb600a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:33.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.248+0000 7ffb7b400640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 0x7ffb600a61c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005fd0 tx=0x7ffb6c009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:33.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.249+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffb64053ff0 con 0x7ffb7c1008a0 2024-04-20T21:43:33.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.482+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 --> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ffb7c02d640 con 0x7ffb600a3d00 2024-04-20T21:43:33.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.499+0000 7ffb78c00640 1 -- 172.21.15.7:0/3674975428 <== mgr.25208 v2:172.21.15.7:6800/2899881467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ffb7c02d640 con 0x7ffb600a3d00 2024-04-20T21:43:33.498 INFO:teuthology.orchestra.run.smithi007.stdout:/ 1m 6m3h 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 msgr2=0x7ffb600a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 0x7ffb600a61c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005fd0 tx=0x7ffb6c009260 comp rx=0 tx=0).stop 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 msgr2=0x7ffb7c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9f0 tx=0x7ffb6400edf0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 shutdown_connections 2024-04-20T21:43:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffb600a3d00 0x7ffb600a61c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7c1008a0 0x7ffb7c101e90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb7c107e80 0x7ffb7c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 --2- 172.21.15.7:0/3674975428 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb7c0ffee0 0x7ffb7c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 >> 172.21.15.7:0/3674975428 conn(0x7ffb7c076ea0 msgr2=0x7ffb7c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 shutdown_connections 2024-04-20T21:43:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:33.502+0000 7ffb80e00640 1 -- 172.21.15.7:0/3674975428 wait complete. 2024-04-20T21:43:33.522 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-20T21:43:33.532 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-20T21:43:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:34 smithi016 ceph-mon[21409]: from='client.27167 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:34 smithi016 ceph-mon[21409]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:34 smithi016 ceph-mon[21409]: from='client.27173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:34.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:34 smithi186 ceph-mon[22157]: from='client.27167 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:34.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:34 smithi186 ceph-mon[22157]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:34.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:34 smithi186 ceph-mon[22157]: from='client.27173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:35.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:34 smithi007 ceph-mon[16867]: from='client.27167 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:35.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:34 smithi007 ceph-mon[16867]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-20T21:43:35.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:34 smithi007 ceph-mon[16867]: from='client.27173 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-20T21:43:36.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:36 smithi016 ceph-mon[21409]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:36 smithi186 ceph-mon[22157]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:37.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:36 smithi007 ceph-mon[16867]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:38.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:38 smithi016 ceph-mon[21409]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:38.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:38 smithi186 ceph-mon[22157]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:39.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:38 smithi007 ceph-mon[16867]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:40 smithi016 ceph-mon[21409]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-20T21:43:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:40 smithi186 ceph-mon[22157]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-20T21:43:41.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:40 smithi007 ceph-mon[16867]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-20T21:43:42.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:42 smithi016 ceph-mon[21409]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:42.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:42 smithi186 ceph-mon[22157]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:43.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:42 smithi007 ceph-mon[16867]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-20T21:43:43.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-20T21:43:43.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-20T21:43:44.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 -- 172.21.15.7:0/1922142726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac8101470 msgr2=0x7f8ac810e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 --2- 172.21.15.7:0/1922142726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac8101470 0x7f8ac810e130 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009900 tx=0x7f8ab002bc40 comp rx=0 tx=0).stop 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 -- 172.21.15.7:0/1922142726 shutdown_connections 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 --2- 172.21.15.7:0/1922142726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac8101470 0x7f8ac810e130 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 --2- 172.21.15.7:0/1922142726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac810e670 0x7f8ac8110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 --2- 172.21.15.7:0/1922142726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac8100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 -- 172.21.15.7:0/1922142726 >> 172.21.15.7:0/1922142726 conn(0x7f8ac8077a10 msgr2=0x7f8ac80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 -- 172.21.15.7:0/1922142726 shutdown_connections 2024-04-20T21:43:44.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.129+0000 7f8acd000640 1 -- 172.21.15.7:0/1922142726 wait complete. 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.131+0000 7f8acd000640 1 Processor -- start 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.131+0000 7f8acd000640 1 -- start start 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 0x7f8ac81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac810e670 0x7f8ac819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8114370 con 0x7f8ac8101470 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81141f0 con 0x7f8ac810e670 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8acd000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81144f0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 0x7f8ac81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48492/0 (socket says 172.21.15.7:48492) 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 -- 172.21.15.7:0/673333781 learned_addr learned my addr 172.21.15.7:0/673333781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 0x7f8ac81a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38906/0 (socket says 172.21.15.7:38906) 2024-04-20T21:43:44.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac810e670 0x7f8ac819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 -- 172.21.15.7:0/673333781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac810e670 msgr2=0x7f8ac819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac810e670 0x7f8ac819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 -- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 msgr2=0x7f8ac81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 0x7f8ac81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.132+0000 7f8ac7400640 1 -- 172.21.15.7:0/673333781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac819afa0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8ac7400640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8abfe00640 1 -- 172.21.15.7:0/673333781 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800ccc0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8abfe00640 1 -- 172.21.15.7:0/673333781 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab8010910 con 0x7f8ac8100ac0 2024-04-20T21:43:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8abfe00640 1 -- 172.21.15.7:0/673333781 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80188a0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac819b290 con 0x7f8ac8100ac0 2024-04-20T21:43:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.133+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac81a75d0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 Processor -- start 2024-04-20T21:43:44.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 -- start start 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8abfe00640 1 -- 172.21.15.7:0/673333781 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8ab8010ac0 con 0x7f8ac8100ac0 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 0x55df3d3f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df3d34f150 con 0x55df3d3f2880 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df3d2f5370 con 0x55df3d3e9380 2024-04-20T21:43:44.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55df3d34b110 con 0x55df3d3e9ca0 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 0x55df3d3f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.137+0000 7f8ac7400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 0x55df3d3f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38910/0 (socket says 172.21.15.7:38910) 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3986838489 learned_addr learned my addr 172.21.15.7:0/3986838489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7400640 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48496/0 (socket says 172.21.15.7:48496) 2024-04-20T21:43:44.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 -- 172.21.15.7:0/3986838489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 msgr2=0x55df3d3f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 -- 172.21.15.7:0/3986838489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 msgr2=0x55df3d3f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 0x55df3d3f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 -- 172.21.15.7:0/3986838489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3d12b7d0 con 0x55df3d3e9380 2024-04-20T21:43:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3e9760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8074d40 tx=0x7f8ac0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f39f5922fa58bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.138+0000 7f8abcc00640 1 -- 172.21.15.7:0/3986838489 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0004d20 con 0x55df3d3e9380 2024-04-20T21:43:44.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8abcc00640 1 -- 172.21.15.7:0/3986838489 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac0004050 con 0x55df3d3e9380 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8abcc00640 1 -- 172.21.15.7:0/3986838489 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0005280 con 0x55df3d3e9380 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8abfe00640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 0x7f8aac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8abfe00640 1 -- 172.21.15.7:0/673333781 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f8ab80d0b80 con 0x7f8ac8100ac0 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ac7400640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 0x7f8aac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3986838489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 msgr2=0x55df3d3e9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3e9760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8074d40 tx=0x7f8ac0011810 comp rx=0 tx=0).stop 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3986838489 shutdown_connections 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3f2880 0x55df3d3f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3986838489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3e9760 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3986838489 >> 172.21.15.7:0/3986838489 conn(0x55df3d3e3ab0 msgr2=0x55df3d3e3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.139+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3986838489 shutdown_connections 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.140+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3986838489 wait complete. 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.140+0000 7f8ac7400640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 0x7f8aac0a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ce3a0 tx=0x7f8ab80d46d0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.140+0000 7f8ad13b0740 1 -- start start 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.140+0000 7f8ad13b0740 1 Processor -- start 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 0x55df3d3ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3efe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df3d3f5670 con 0x55df3d3e9380 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df3d308740 con 0x55df3d3f2880 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x55df3d3f57f0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ad13b0740 10 client.0 _ll_register_callbacks cb 0x55df3d303b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ac7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 0x55df3d3ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ac7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 0x55df3d3ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38918/0 (socket says 172.21.15.7:38918) 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ac7e00640 1 -- 172.21.15.7:0/3669503860 learned_addr learned my addr 172.21.15.7:0/3669503860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3efe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.141+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 msgr2=0x55df3d3f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3669503860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 msgr2=0x55df3d3ef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 0x55df3d3ef900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3669503860 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3d0613f0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3efe40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005ec0 tx=0x7f8ab0009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e041a5126b15ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ab5600640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00046c0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f8ab0004860 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.142+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac0004050 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ab0004a60 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 10 client.27130 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df3d0210f0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac0004050 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8ab004f020 con 0x55df3d3e9ca0 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 10 client.27130 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:44.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 10 client.27130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 20 client.27130 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d402ba0 0x55df3d404f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 4 client.27130 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.143+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3d35cb70 con 0x55df3d402ba0 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.144+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d402ba0 0x55df3d404f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.144+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d402ba0 0x55df3d404f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209b96258acf3c16 server_cookie=34b303a9569f84da in_seq=0 out_seq=0 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.144+0000 7f8ab5600640 10 client.27130 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:44.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ab5600640 1 -- 172.21.15.7:0/3669503860 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df3d35cb70 con 0x55df3d402ba0 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ab5600640 10 client.27130 handle_command_reply: tid=0 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 client.27130 shutdown 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 msgr2=0x55df3d3efe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3efe40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005ec0 tx=0x7f8ab0009880 comp rx=0 tx=0).stop 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 shutdown_connections 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d402ba0 0x55df3d404f90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.145+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9ca0 0x55df3d3efe40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3669503860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9380 0x55df3d3ef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ab5600640 0 client.27130 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 >> 172.21.15.7:0/3669503860 conn(0x55df3d3e3ab0 msgr2=0x55df3d3e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 shutdown_connections 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3669503860 wait complete. 2024-04-20T21:43:44.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.146+0000 7f8ad13b0740 20 client.27130 trim_cache size 0 max 16384 2024-04-20T21:43:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.165+0000 7f8ad13b0740 1 Processor -- start 2024-04-20T21:43:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.165+0000 7f8ad13b0740 1 -- start start 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3f2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 0x55df3d3ee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81b2670 con 0x55df3d3e9ca0 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81aff30 con 0x55df3d3e9380 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81adda0 con 0x55df3d3f2880 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ac7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ac7e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 0x55df3d3ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38934/0 (socket says 172.21.15.7:38934) 2024-04-20T21:43:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.166+0000 7f8ac6a00640 1 -- 172.21.15.7:0/353050899 learned_addr learned my addr 172.21.15.7:0/353050899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 -- 172.21.15.7:0/353050899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 msgr2=0x55df3d3ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 0x55df3d3ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 -- 172.21.15.7:0/353050899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 msgr2=0x55df3d3f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 -- 172.21.15.7:0/353050899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3d3e8d50 con 0x55df3d3e9ca0 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 0x55df3d3ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ab002c180 tx=0x7f8ab002bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f1e46a17cff15a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ab7e00640 1 -- 172.21.15.7:0/353050899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00035f0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ab7e00640 1 -- 172.21.15.7:0/353050899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab0043070 con 0x55df3d3e9ca0 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.167+0000 7f8ab7e00640 1 -- 172.21.15.7:0/353050899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0034440 con 0x55df3d3e9ca0 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 -- 172.21.15.7:0/353050899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 msgr2=0x55df3d3fe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8ab002c180 tx=0x7f8ab002bc40 comp rx=0 tx=0).stop 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 -- 172.21.15.7:0/353050899 shutdown_connections 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3f2880 0x55df3d3ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3fe530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/353050899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3e9380 0x55df3d3f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 -- 172.21.15.7:0/353050899 >> 172.21.15.7:0/353050899 conn(0x55df3d3e3ab0 msgr2=0x55df3d3e5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 -- 172.21.15.7:0/353050899 shutdown_connections 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.168+0000 7f8ad13b0740 1 -- 172.21.15.7:0/353050899 wait complete. 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 -- start start 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 Processor -- start 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9380 0x55df3d3faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81ae3f0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81af860 con 0x55df3d3f2880 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac81aeac0 con 0x55df3d3e9380 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.169+0000 7f8ad13b0740 10 client.0 _ll_register_callbacks cb 0x7f8ab0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38944/0 (socket says 172.21.15.7:38944) 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3772074792 learned_addr learned my addr 172.21.15.7:0/3772074792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9380 0x55df3d3faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3772074792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9380 msgr2=0x55df3d3faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9380 0x55df3d3faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 msgr2=0x55df3d3f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 -- 172.21.15.7:0/3772074792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3d3f82c0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ac6a00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55df3d3f0a90 tx=0x7f8ab0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7c0276ef1dfa1f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.170+0000 7f8ab5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:43:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0034640 con 0x55df3d3e9ca0 2024-04-20T21:43:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f8ab00348d0 con 0x55df3d3e9ca0 2024-04-20T21:43:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ab0043070 con 0x55df3d3e9ca0 2024-04-20T21:43:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8ab0067050 con 0x55df3d3e9ca0 2024-04-20T21:43:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ad13b0740 10 client.16953 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55df3d469c20 con 0x55df3d3e9ca0 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.171+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ab0043070 con 0x55df3d3e9ca0 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8ab004f070 con 0x55df3d3e9ca0 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 10 client.16953 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 20 client.16953 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d46a2d0 0x55df3d46c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 4 client.16953 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:43:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df3d3569e0 con 0x55df3d46a2d0 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.172+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d46a2d0 0x55df3d46c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ac7e00640 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d46a2d0 0x55df3d46c6f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6835a577932fae2e server_cookie=e8e80b5890e5521f in_seq=0 out_seq=0 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ab5600640 10 client.16953 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ab5600640 1 -- 172.21.15.7:0/3772074792 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df3d3569e0 con 0x55df3d46a2d0 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ab5600640 10 client.16953 handle_command_reply: tid=0 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 client.16953 shutdown 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 msgr2=0x55df3d3faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55df3d3f0a90 tx=0x7f8ab0003a80 comp rx=0 tx=0).stop 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 shutdown_connections 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df3d3f2880 0x55df3d3f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x55df3d46a2d0 0x55df3d46c6f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df3d3e9ca0 0x55df3d3faf80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 --2- 172.21.15.7:0/3772074792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55df3d3e9380 0x55df3d3faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ab5600640 0 client.16953 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 >> 172.21.15.7:0/3772074792 conn(0x55df3d3e3ab0 msgr2=0x55df3d3e2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 shutdown_connections 2024-04-20T21:43:44.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 1 -- 172.21.15.7:0/3772074792 wait complete. 2024-04-20T21:43:44.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.173+0000 7f8ad13b0740 20 client.16953 trim_cache size 0 max 16384 2024-04-20T21:43:44.180 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-04-20T21:43:44.180 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {} 2024-04-20T21:43:44.181 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:44.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 msgr2=0x7f8aac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 0x7f8aac0a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80ce3a0 tx=0x7f8ab80d46d0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 msgr2=0x7f8ac81a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800edf0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 shutdown_connections 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f8aac0a3da0 0x7f8aac0a6260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ac810e670 0x7f8ac819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ac8101470 0x7f8ac81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 --2- 172.21.15.7:0/673333781 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8ac8100ac0 0x7f8ac81a05e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.174+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 >> 172.21.15.7:0/673333781 conn(0x7f8ac8077a10 msgr2=0x7f8ac80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.175+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 shutdown_connections 2024-04-20T21:43:44.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.175+0000 7f8acd000640 1 -- 172.21.15.7:0/673333781 wait complete. 2024-04-20T21:43:44.193 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:43:44.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 -- 172.21.15.7:0/1500983869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3901042a0 msgr2=0x7fb390104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 --2- 172.21.15.7:0/1500983869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3901042a0 0x7fb390104680 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb380009900 tx=0x7fb38002bc30 comp rx=0 tx=0).stop 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 -- 172.21.15.7:0/1500983869 shutdown_connections 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 --2- 172.21.15.7:0/1500983869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390109370 0x7fb390110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 --2- 172.21.15.7:0/1500983869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb390104c50 0x7fb390108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 --2- 172.21.15.7:0/1500983869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3901042a0 0x7fb390104680 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 -- 172.21.15.7:0/1500983869 >> 172.21.15.7:0/1500983869 conn(0x7fb3900ffae0 msgr2=0x7fb390101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 -- 172.21.15.7:0/1500983869 shutdown_connections 2024-04-20T21:43:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.412+0000 7fb38fe00640 1 -- 172.21.15.7:0/1500983869 wait complete. 2024-04-20T21:43:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.413+0000 7fb38fe00640 1 Processor -- start 2024-04-20T21:43:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.413+0000 7fb38fe00640 1 -- start start 2024-04-20T21:43:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3901042a0 0x7fb3901a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb39019a810 0x7fb39019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb390114450 con 0x7fb39019a810 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901142d0 con 0x7fb390109370 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901145d0 con 0x7fb3901042a0 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb386000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb39019a810 0x7fb39019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb386000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37626/0 (socket says 172.21.15.7:37626) 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb38ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3901042a0 0x7fb3901a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.414+0000 7fb386000640 1 -- 172.21.15.7:0/3079667439 learned_addr learned my addr 172.21.15.7:0/3079667439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 -- 172.21.15.7:0/3079667439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3901042a0 msgr2=0x7fb3901a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3901042a0 0x7fb3901a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 -- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb39019a810 msgr2=0x7fb39019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb39019a810 0x7fb39019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 -- 172.21.15.7:0/3079667439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39019b3a0 con 0x7fb390109370 2024-04-20T21:43:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb386000640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb37800e9c0 tx=0x7fb37800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800ccc0 con 0x7fb390109370 2024-04-20T21:43:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378005500 con 0x7fb390109370 2024-04-20T21:43:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39019b690 con 0x7fb390109370 2024-04-20T21:43:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.415+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378010430 con 0x7fb390109370 2024-04-20T21:43:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.416+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3901a7820 con 0x7fb390109370 2024-04-20T21:43:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.417+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb390105770 con 0x7fb390109370 2024-04-20T21:43:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.419+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fb378005000 con 0x7fb390109370 2024-04-20T21:43:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.421+0000 7fb387e00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 0x7fb3740a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.421+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fb3780d0c60 con 0x7fb390109370 2024-04-20T21:43:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.421+0000 7fb38ea00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 0x7fb3740a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.422+0000 7fb38ea00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 0x7fb3740a61d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb380000c00 tx=0x7fb38002df90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.425+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fb39019b690 con 0x7fb390109370 2024-04-20T21:43:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.679+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb39010c5a0 con 0x7fb390109370 2024-04-20T21:43:44.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.680+0000 7fb387e00640 1 -- 172.21.15.7:0/3079667439 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7fb37809d2c0 con 0x7fb390109370 2024-04-20T21:43:44.679 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:43:44.680 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:43:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:43:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 msgr2=0x7fb3740a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 0x7fb3740a61d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb380000c00 tx=0x7fb38002df90 comp rx=0 tx=0).stop 2024-04-20T21:43:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 msgr2=0x7fb3901a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb37800e9c0 tx=0x7fb37800edc0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 shutdown_connections 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fb3740a3d10 0x7fb3740a61d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390109370 0x7fb3901a0b90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb39019a810 0x7fb39019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 --2- 172.21.15.7:0/3079667439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3901042a0 0x7fb3901a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.683+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 >> 172.21.15.7:0/3079667439 conn(0x7fb3900ffae0 msgr2=0x7fb390101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.684+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 shutdown_connections 2024-04-20T21:43:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.684+0000 7fb38fe00640 1 -- 172.21.15.7:0/3079667439 wait complete. 2024-04-20T21:43:44.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 -- 172.21.15.7:0/3641523164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac103860 msgr2=0x7f61ac10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 --2- 172.21.15.7:0/3641523164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac103860 0x7f61ac10bd40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6194009900 tx=0x7f619402bcc0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 -- 172.21.15.7:0/3641523164 shutdown_connections 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 --2- 172.21.15.7:0/3641523164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac10c360 0x7f61ac10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 --2- 172.21.15.7:0/3641523164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac103860 0x7f61ac10bd40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 --2- 172.21.15.7:0/3641523164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac102f40 0x7f61ac103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 -- 172.21.15.7:0/3641523164 >> 172.21.15.7:0/3641523164 conn(0x7f61ac077a10 msgr2=0x7f61ac0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 -- 172.21.15.7:0/3641523164 shutdown_connections 2024-04-20T21:43:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.918+0000 7f61b2000640 1 -- 172.21.15.7:0/3641523164 wait complete. 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 Processor -- start 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 -- start start 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac102f40 0x7f61ac1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac103860 0x7f61ac198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac112140 con 0x7f61ac102f40 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac111fc0 con 0x7f61ac10c360 2024-04-20T21:43:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.919+0000 7f61b2000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1122c0 con 0x7f61ac103860 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac102f40 0x7f61ac1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37650/0 (socket says 172.21.15.7:37650) 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 -- 172.21.15.7:0/140001055 learned_addr learned my addr 172.21.15.7:0/140001055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61abe00640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac103860 0x7f61ac198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 -- 172.21.15.7:0/140001055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac103860 msgr2=0x7f61ac198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac103860 0x7f61ac198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 -- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac102f40 msgr2=0x7f61ac1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac102f40 0x7f61ac1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 -- 172.21.15.7:0/140001055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac19d6d0 con 0x7f61ac10c360 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.920+0000 7f61b1600640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e9b0 tx=0x7f61a400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.921+0000 7f61a9600640 1 -- 172.21.15.7:0/140001055 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a400cd80 con 0x7f61ac10c360 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.921+0000 7f61a9600640 1 -- 172.21.15.7:0/140001055 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a40035f0 con 0x7f61ac10c360 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.921+0000 7f61a9600640 1 -- 172.21.15.7:0/140001055 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4010670 con 0x7f61ac10c360 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.921+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac19d9c0 con 0x7f61ac10c360 2024-04-20T21:43:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.921+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1a5250 con 0x7f61ac10c360 2024-04-20T21:43:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.925+0000 7f61b6399740 1 Processor -- start 2024-04-20T21:43:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61a9600640 1 -- 172.21.15.7:0/140001055 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f61a40037a0 con 0x7f61ac10c360 2024-04-20T21:43:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 -- start start 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 0x564c34aa5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 0x564c34ab3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34ab5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564c34a0b050 con 0x564c34aa6500 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c349b1490 con 0x564c34aa5b50 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.926+0000 7f61b6399740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564c34a07010 con 0x564c34ab3a80 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 0x564c34aa5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 0x564c34ab3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 0x564c34aa5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37652/0 (socket says 172.21.15.7:37652) 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61abe00640 1 -- 172.21.15.7:0/4042623987 learned_addr learned my addr 172.21.15.7:0/4042623987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61abe00640 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34ab5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 -- 172.21.15.7:0/4042623987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 msgr2=0x564c34ab5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34ab5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 -- 172.21.15.7:0/4042623987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 msgr2=0x564c34aa5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 0x564c34aa5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61b0c00640 1 -- 172.21.15.7:0/4042623987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c349b9070 con 0x564c34aa6500 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.927+0000 7f61a9600640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 0x7f61900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61b0c00640 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 0x564c34ab3540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f619c009fa0 tx=0x7f619c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb88b139a85df29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61a9600640 1 -- 172.21.15.7:0/140001055 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f61a40d0a80 con 0x7f61ac10c360 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61abe00640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 0x7f61900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61a1c00640 1 -- 172.21.15.7:0/4042623987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c003c30 con 0x564c34aa6500 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61a1c00640 1 -- 172.21.15.7:0/4042623987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c003dd0 con 0x564c34aa6500 2024-04-20T21:43:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.928+0000 7f61a1c00640 1 -- 172.21.15.7:0/4042623987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00fa30 con 0x564c34aa6500 2024-04-20T21:43:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- 172.21.15.7:0/4042623987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 msgr2=0x564c34ab3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 0x564c34ab3540 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f619c009fa0 tx=0x7f619c0118d0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61abe00640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 0x7f61900a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x564c34aa1cc0 tx=0x7f6194005b50 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- 172.21.15.7:0/4042623987 shutdown_connections 2024-04-20T21:43:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34ab5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa6500 0x564c34ab3540 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 --2- 172.21.15.7:0/4042623987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa5b50 0x564c34aa5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- 172.21.15.7:0/4042623987 >> 172.21.15.7:0/4042623987 conn(0x564c34aa08a0 msgr2=0x564c34aa0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- 172.21.15.7:0/4042623987 shutdown_connections 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- 172.21.15.7:0/4042623987 wait complete. 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.929+0000 7f61b6399740 1 -- start start 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 Processor -- start 2024-04-20T21:43:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34ab0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34ab3a80 0x564c34ab14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564c34ab6840 con 0x564c34ab3a80 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c349c4860 con 0x564c34aa6500 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x564c34ab69c0 con 0x564c34aa5b50 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.930+0000 7f61b6399740 10 client.0 _ll_register_callbacks cb 0x564c34992fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61abe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34ab0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48584/0 (socket says 172.21.15.7:48584) 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 -- 172.21.15.7:0/3695365036 learned_addr learned my addr 172.21.15.7:0/3695365036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b1600640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34ab3a80 0x564c34ab14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 -- 172.21.15.7:0/3695365036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 msgr2=0x564c34ab0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34ab0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 -- 172.21.15.7:0/3695365036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34ab3a80 msgr2=0x564c34ab14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34ab3a80 0x564c34ab14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 -- 172.21.15.7:0/3695365036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c349c8950 con 0x564c34aa5b50 2024-04-20T21:43:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.931+0000 7f61b0c00640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f619c01b040 tx=0x7f619c004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3711edec182d54f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f619a000640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c028020 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f619c003dd0 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f619c00f9e0 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f619c033050 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f61b6399740 10 client.27154 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.932+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c34996820 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c003dd0 con 0x564c34aa5b50 2024-04-20T21:43:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f619c005040 con 0x564c34aa5b50 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 10 client.27154 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 10 client.27154 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 20 client.27154 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34ac39b0 0x564c34ac5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 4 client.27154 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c34a18a70 con 0x564c34ac39b0 2024-04-20T21:43:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.933+0000 7f61abe00640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34ac39b0 0x564c34ac5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.934+0000 7f61abe00640 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34ac39b0 0x564c34ac5da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75951286c59430e1 server_cookie=ebf993da6cd150d9 in_seq=0 out_seq=0 2024-04-20T21:43:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.934+0000 7f619a000640 10 client.27154 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f619a000640 1 -- 172.21.15.7:0/3695365036 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c34a18a70 con 0x564c34ac39b0 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f619a000640 10 client.27154 handle_command_reply: tid=0 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 client.27154 shutdown 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 msgr2=0x564c34ab0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f619c01b040 tx=0x7f619c004cf0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 shutdown_connections 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34ac39b0 0x564c34ac5da0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34ab0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34ab3a80 0x564c34ab14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 --2- 172.21.15.7:0/3695365036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34aa5b50 0x564c34ab0a60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 >> 172.21.15.7:0/3695365036 conn(0x564c34aa08a0 msgr2=0x564c34aa25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f619a000640 0 client.27154 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 shutdown_connections 2024-04-20T21:43:44.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 1 -- 172.21.15.7:0/3695365036 wait complete. 2024-04-20T21:43:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.935+0000 7f61b6399740 20 client.27154 trim_cache size 0 max 16384 2024-04-20T21:43:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:44 smithi016 ceph-mon[21409]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.950+0000 7f61b6399740 1 Processor -- start 2024-04-20T21:43:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.950+0000 7f61b6399740 1 -- start start 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34aa27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34abf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1ae220 con 0x564c34aa5b50 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1abae0 con 0x564c34aa6500 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b6399740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1a9950 con 0x564c34ab3a80 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39022/0 (socket says 172.21.15.7:39022) 2024-04-20T21:43:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 -- 172.21.15.7:0/1262240849 learned_addr learned my addr 172.21.15.7:0/1262240849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b1600640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34abf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61abe00640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34aa27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 -- 172.21.15.7:0/1262240849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 msgr2=0x564c34abf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34abf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 -- 172.21.15.7:0/1262240849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 msgr2=0x564c34aa27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34aa27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b0c00640 1 -- 172.21.15.7:0/1262240849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c3471d3f0 con 0x564c34aa5b50 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.951+0000 7f61b1600640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34abf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.952+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f619c005490 tx=0x7f619c00f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1577fa789a06004 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.952+0000 7f61a1200640 1 -- 172.21.15.7:0/1262240849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c03c070 con 0x564c34aa5b50 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.952+0000 7f61a1200640 1 -- 172.21.15.7:0/1262240849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c00f600 con 0x564c34aa5b50 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.952+0000 7f61a1200640 1 -- 172.21.15.7:0/1262240849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c028040 con 0x564c34aa5b50 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- 172.21.15.7:0/1262240849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 msgr2=0x564c34ab3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f619c005490 tx=0x7f619c00f4c0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- 172.21.15.7:0/1262240849 shutdown_connections 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34aa27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34abf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 --2- 172.21.15.7:0/1262240849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34ab3190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- 172.21.15.7:0/1262240849 >> 172.21.15.7:0/1262240849 conn(0x564c34aa08a0 msgr2=0x564c34a9f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- 172.21.15.7:0/1262240849 shutdown_connections 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- 172.21.15.7:0/1262240849 wait complete. 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 -- start start 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.953+0000 7f61b6399740 1 Processor -- start 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34abc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34aad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1a9fa0 con 0x564c34aa5b50 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1ab410 con 0x564c34aa6500 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1aa670 con 0x564c34ab3a80 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b6399740 10 client.0 _ll_register_callbacks cb 0x7f619c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39036/0 (socket says 172.21.15.7:39036) 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b0c00640 1 -- 172.21.15.7:0/1806046585 learned_addr learned my addr 172.21.15.7:0/1806046585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:44.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61b1600640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34aad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.954+0000 7f61abe00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34abc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 -- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 msgr2=0x564c34aad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34aad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 -- 172.21.15.7:0/1806046585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 msgr2=0x564c34abc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34abc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 -- 172.21.15.7:0/1806046585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c347e77d0 con 0x564c34aa5b50 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b0c00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e28d41e7c5a4316 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f619a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:43:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c03c070 con 0x564c34aa5b50 2024-04-20T21:43:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f619c004cf0 con 0x564c34aa5b50 2024-04-20T21:43:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.955+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f619c00f600 con 0x564c34aa5b50 2024-04-20T21:43:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f619c028070 con 0x564c34aa5b50 2024-04-20T21:43:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 10 client.16977 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c34b29820 con 0x564c34aa5b50 2024-04-20T21:43:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c00f600 con 0x564c34aa5b50 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f619c047020 con 0x564c34aa5b50 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 10 client.16977 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 10 client.16977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 20 client.16977 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34b29e90 0x564c34b2c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 4 client.16977 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.956+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 --> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c34a128e0 con 0x564c34b29e90 2024-04-20T21:43:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.957+0000 7f61abe00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34b29e90 0x564c34b2c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.957+0000 7f61abe00640 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34b29e90 0x564c34b2c280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9ed7881a352dc05 server_cookie=dd56a53b519ca93a in_seq=0 out_seq=0 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.957+0000 7f619a000640 10 client.16977 ms_handle_connect on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f619a000640 1 -- 172.21.15.7:0/1806046585 <== mds.2 v2:172.21.15.186:6836/2315081064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c34a128e0 con 0x564c34b29e90 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f619a000640 10 client.16977 handle_command_reply: tid=0 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 client.16977 shutdown 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 msgr2=0x564c34abbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f619c006010 tx=0x7f619c00ff20 comp rx=0 tx=0).stop 2024-04-20T21:43:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 shutdown_connections 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:6836/2315081064,v1:172.21.15.186:6837/2315081064] conn(0x564c34b29e90 0x564c34b2c280 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c34aa6500 0x564c34abc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564c34ab3a80 0x564c34aad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 --2- 172.21.15.7:0/1806046585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564c34aa5b50 0x564c34abbc80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f619a000640 0 client.16977 ms_handle_reset on v2:172.21.15.186:6836/2315081064 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 >> 172.21.15.7:0/1806046585 conn(0x564c34aa08a0 msgr2=0x564c34aaf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 shutdown_connections 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 1 -- 172.21.15.7:0/1806046585 wait complete. 2024-04-20T21:43:44.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.958+0000 7f61b6399740 20 client.16977 trim_cache size 0 max 16384 2024-04-20T21:43:44.965 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 msgr2=0x7f61900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 0x7f61900a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x564c34aa1cc0 tx=0x7f6194005b50 comp rx=0 tx=0).stop 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 msgr2=0x7f61ac19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e9b0 tx=0x7f61a400ee70 comp rx=0 tx=0).stop 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 shutdown_connections 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61900a3d20 0x7f61900a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61ac10c360 0x7f61ac19cfa0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f61ac103860 0x7f61ac198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 --2- 172.21.15.7:0/140001055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61ac102f40 0x7f61ac1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 >> 172.21.15.7:0/140001055 conn(0x7f61ac077a10 msgr2=0x7f61ac0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 shutdown_connections 2024-04-20T21:43:44.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:44.959+0000 7f61b2000640 1 -- 172.21.15.7:0/140001055 wait complete. 2024-04-20T21:43:44.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:44 smithi186 ceph-mon[22157]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:44.973 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:44 smithi007 ceph-mon[16867]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:44.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-20T21:43:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 -- 172.21.15.7:0/379695415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a08102f40 msgr2=0x7f6a08103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/379695415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a08102f40 0x7f6a08103320 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009ca0 tx=0x7f69f802c180 comp rx=0 tx=0).stop 2024-04-20T21:43:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 -- 172.21.15.7:0/379695415 shutdown_connections 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/379695415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a0810c360 0x7f6a0810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/379695415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 0x7f6a0810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/379695415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a08102f40 0x7f6a08103320 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 -- 172.21.15.7:0/379695415 >> 172.21.15.7:0/379695415 conn(0x7f6a08077a10 msgr2=0x7f6a080fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 -- 172.21.15.7:0/379695415 shutdown_connections 2024-04-20T21:43:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.197+0000 7f6a0ea00640 1 -- 172.21.15.7:0/379695415 wait complete. 2024-04-20T21:43:45.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.198+0000 7f6a0ea00640 1 Processor -- start 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 -- start start 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 0x7f6a081a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 0x7f6a081a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08112110 con 0x7f6a0810c360 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08111f90 con 0x7f6a08102f40 2024-04-20T21:43:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0ea00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a08112290 con 0x7f6a08103860 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39050/0 (socket says 172.21.15.7:39050) 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 0x7f6a081a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0e000640 1 -- 172.21.15.7:0/4251898975 learned_addr learned my addr 172.21.15.7:0/4251898975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.199+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 0x7f6a081a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 -- 172.21.15.7:0/4251898975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 msgr2=0x7f6a081a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 0x7f6a081a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 -- 172.21.15.7:0/4251898975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 msgr2=0x7f6a081a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 0x7f6a081a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 -- 172.21.15.7:0/4251898975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0819b050 con 0x7f6a0810c360 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0d600640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 0x7f6a081a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a0e000640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9b0 tx=0x7f6a0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a06000640 1 -- 172.21.15.7:0/4251898975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000cd80 con 0x7f6a0810c360 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.200+0000 7f6a06000640 1 -- 172.21.15.7:0/4251898975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a000035f0 con 0x7f6a0810c360 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.201+0000 7f6a06000640 1 -- 172.21.15.7:0/4251898975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00010670 con 0x7f6a0810c360 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.201+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0819b340 con 0x7f6a0810c360 2024-04-20T21:43:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.201+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a081a7560 con 0x7f6a0810c360 2024-04-20T21:43:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.205+0000 7f6a12d84740 1 Processor -- start 2024-04-20T21:43:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.205+0000 7f6a06000640 1 -- 172.21.15.7:0/4251898975 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f6a000037a0 con 0x7f6a0810c360 2024-04-20T21:43:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 -- start start 2024-04-20T21:43:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf955130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf955670 0x556bdf957a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bdf8b1050 con 0x556bdf94c170 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556bdf857490 con 0x556bdf955670 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a12d84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556bdf8ad010 con 0x556bdf94ca90 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a0cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a0d600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf955130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a0cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39060/0 (socket says 172.21.15.7:39060) 2024-04-20T21:43:45.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.206+0000 7f6a0cc00640 1 -- 172.21.15.7:0/1524468572 learned_addr learned my addr 172.21.15.7:0/1524468572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf955670 0x556bdf957a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 -- 172.21.15.7:0/1524468572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf955670 msgr2=0x556bdf957a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf955670 0x556bdf957a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 -- 172.21.15.7:0/1524468572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 msgr2=0x556bdf94c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 -- 172.21.15.7:0/1524468572 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdf85f070 con 0x556bdf94ca90 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a0d600640 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf955130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006010 tx=0x7f69f80098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5939050a781136a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.207+0000 7f6a06000640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 0x7f69ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f69fe600640 1 -- 172.21.15.7:0/1524468572 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8043070 con 0x556bdf94ca90 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f69fe600640 1 -- 172.21.15.7:0/1524468572 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8003e50 con 0x556bdf94ca90 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f69fe600640 1 -- 172.21.15.7:0/1524468572 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8039040 con 0x556bdf94ca90 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f6a06000640 1 -- 172.21.15.7:0/4251898975 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f6a000d0a80 con 0x7f6a0810c360 2024-04-20T21:43:45.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 0x7f69ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f6a12d84740 1 -- 172.21.15.7:0/1524468572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 msgr2=0x556bdf955130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.208+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf955130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006010 tx=0x7f69f80098c0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 0x7f69ec0a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x556bdf947a30 tx=0x7f69f4009830 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 -- 172.21.15.7:0/1524468572 shutdown_connections 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf955670 0x556bdf957a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf955130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1524468572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf94c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 -- 172.21.15.7:0/1524468572 >> 172.21.15.7:0/1524468572 conn(0x556bdf9468a0 msgr2=0x556bdf946c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 -- 172.21.15.7:0/1524468572 shutdown_connections 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 -- 172.21.15.7:0/1524468572 wait complete. 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 -- start start 2024-04-20T21:43:45.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.209+0000 7f6a12d84740 1 Processor -- start 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 0x556bdf9526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 0x556bdf953120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556bdf958400 con 0x556bdf955670 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556bdf86a860 con 0x556bdf94c170 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x556bdf958580 con 0x556bdf94ca90 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a12d84740 10 client.0 _ll_register_callbacks cb 0x556bdf838fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:45.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a0cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 0x556bdf953120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a0cc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48620/0 (socket says 172.21.15.7:48620) 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a0cc00640 1 -- 172.21.15.7:0/2551602237 learned_addr learned my addr 172.21.15.7:0/2551602237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.210+0000 7f6a0d600640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 0x556bdf9526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 -- 172.21.15.7:0/2551602237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 msgr2=0x556bdf9526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 0x556bdf9526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 -- 172.21.15.7:0/2551602237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 msgr2=0x556bdf953120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 0x556bdf953120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 -- 172.21.15.7:0/2551602237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bdf86e950 con 0x556bdf94ca90 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0d600640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 0x556bdf9526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0e000640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 0x556bdf953120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f69f400fc10 tx=0x7f69f4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4759bf9f0d0110bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.211+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4004460 con 0x556bdf94ca90 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f8003e50 con 0x556bdf94ca90 2024-04-20T21:43:45.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mon.1 v2:172.21.15.186:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f69f40229c0 con 0x556bdf94ca90 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69f4022bc0 con 0x556bdf94ca90 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f6a12d84740 10 client.27172 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bdf83c820 con 0x556bdf94ca90 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.212+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mon.1 v2:172.21.15.186:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69f8003e50 con 0x556bdf94ca90 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mon.1 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f69f402b5d0 con 0x556bdf94ca90 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 10 client.27172 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 10 client.27172 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:43:45.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 20 client.27172 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf965600 0x556bdf967aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 4 client.27172 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bdf8bea70 con 0x556bdf965600 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.213+0000 7f6a0d600640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf965600 0x556bdf967aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.214+0000 7f6a0d600640 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf965600 0x556bdf967aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a7df2417b61e509 server_cookie=a7d8f4c309f942ba in_seq=0 out_seq=0 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.214+0000 7f69f2a00640 10 client.27172 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.214+0000 7f69f2a00640 1 -- 172.21.15.7:0/2551602237 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bdf8bea70 con 0x556bdf965600 2024-04-20T21:43:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.214+0000 7f69f2a00640 10 client.27172 handle_command_reply: tid=0 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 client.27172 shutdown 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 msgr2=0x556bdf952be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f69f400fc10 tx=0x7f69f4004730 comp rx=0 tx=0).stop 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 shutdown_connections 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf965600 0x556bdf967aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf955670 0x556bdf953120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94ca90 0x556bdf952be0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f69f2a00640 0 client.27172 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 --2- 172.21.15.7:0/2551602237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94c170 0x556bdf9526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 >> 172.21.15.7:0/2551602237 conn(0x556bdf9468a0 msgr2=0x556bdf9485f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 shutdown_connections 2024-04-20T21:43:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 1 -- 172.21.15.7:0/2551602237 wait complete. 2024-04-20T21:43:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.215+0000 7f6a12d84740 20 client.27172 trim_cache size 0 max 16384 2024-04-20T21:43:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 Processor -- start 2024-04-20T21:43:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 -- start start 2024-04-20T21:43:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 0x556bdf954dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94c170 0x556bdf9612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.230+0000 7f6a12d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081b24d0 con 0x556bdf94ca90 2024-04-20T21:43:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a12d84740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081afd90 con 0x556bdf9468a0 2024-04-20T21:43:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a12d84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081adc00 con 0x556bdf94c170 2024-04-20T21:43:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39080/0 (socket says 172.21.15.7:39080) 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 -- 172.21.15.7:0/1768405599 learned_addr learned my addr 172.21.15.7:0/1768405599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0d600640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 0x556bdf954dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94c170 0x556bdf9612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 -- 172.21.15.7:0/1768405599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94c170 msgr2=0x556bdf9612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94c170 0x556bdf9612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 -- 172.21.15.7:0/1768405599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 msgr2=0x556bdf954dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 0x556bdf954dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0e000640 1 -- 172.21.15.7:0/1768405599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bdf5c33f0 con 0x556bdf94ca90 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.231+0000 7f6a0d600640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 0x556bdf954dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.232+0000 7f6a0e000640 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a0009c6e0 tx=0x7f6a0009c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7523d26b310a9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.232+0000 7f69fdc00640 1 -- 172.21.15.7:0/1768405599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000a0d90 con 0x556bdf94ca90 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.232+0000 7f69fdc00640 1 -- 172.21.15.7:0/1768405599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a000a5650 con 0x556bdf94ca90 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.232+0000 7f69fdc00640 1 -- 172.21.15.7:0/1768405599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000870b0 con 0x556bdf94ca90 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- 172.21.15.7:0/1768405599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 msgr2=0x556bdf952d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a0009c6e0 tx=0x7f6a0009c640 comp rx=0 tx=0).stop 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- 172.21.15.7:0/1768405599 shutdown_connections 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94ca90 0x556bdf952d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf94c170 0x556bdf9612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 --2- 172.21.15.7:0/1768405599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf9468a0 0x556bdf954dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- 172.21.15.7:0/1768405599 >> 172.21.15.7:0/1768405599 conn(0x556bdf94e070 msgr2=0x556bdf952660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- 172.21.15.7:0/1768405599 shutdown_connections 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- 172.21.15.7:0/1768405599 wait complete. 2024-04-20T21:43:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 -- start start 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.233+0000 7f6a12d84740 1 Processor -- start 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 0x556bdf959290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf9597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 0x556bdf959d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081ae250 con 0x556bdf94c170 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081af6c0 con 0x556bdf94ca90 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f6a081ae920 con 0x556bdf9468a0 2024-04-20T21:43:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a12d84740 10 client.0 _ll_register_callbacks cb 0x7f69f4010e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a0cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf9597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a0d600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 0x556bdf959290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a0d600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 0x556bdf959290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48632/0 (socket says 172.21.15.7:48632) 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a0d600640 1 -- 172.21.15.7:0/3395228644 learned_addr learned my addr 172.21.15.7:0/3395228644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.234+0000 7f6a0e000640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 0x556bdf959d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 -- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 msgr2=0x556bdf959290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 0x556bdf959290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 -- 172.21.15.7:0/3395228644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 msgr2=0x556bdf959d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 0x556bdf959d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 -- 172.21.15.7:0/3395228644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bdf68d7d0 con 0x556bdf94c170 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0e000640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 0x556bdf959d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a0cc00640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf9597d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f69f4010010 tx=0x7f69f400f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7555242e7c57dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4040020 con 0x556bdf94c170 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a000a5650 con 0x556bdf94c170 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f69f402b3f0 con 0x556bdf94c170 2024-04-20T21:43:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.235+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69f4038040 con 0x556bdf94c170 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 10 client.16995 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bdf9cbda0 con 0x556bdf94c170 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a000a5650 con 0x556bdf94c170 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f69f403b070 con 0x556bdf94c170 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 10 client.16995 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 10 client.16995 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 20 client.16995 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf955670 0x556bdf9ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 4 client.16995 mds_command: new command op to 24559 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.236+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 --> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bdf8b88e0 con 0x556bdf955670 2024-04-20T21:43:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f6a0d600640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf955670 0x556bdf9ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f6a0d600640 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf955670 0x556bdf9ce4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a203d00dc893bfc7 server_cookie=6471c3183fb95c36 in_seq=0 out_seq=0 2024-04-20T21:43:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f69f2a00640 10 client.16995 ms_handle_connect on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f69f2a00640 1 -- 172.21.15.7:0/3395228644 <== mds.1 v2:172.21.15.186:6838/2448649165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bdf8b88e0 con 0x556bdf955670 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f69f2a00640 10 client.16995 handle_command_reply: tid=0 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.237+0000 7f6a12d84740 1 client.16995 shutdown 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 msgr2=0x556bdf9597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf9597d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f69f4010010 tx=0x7f69f400f7e0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 shutdown_connections 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:6838/2448649165,v1:172.21.15.186:6839/2448649165] conn(0x556bdf955670 0x556bdf9ce4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556bdf94ca90 0x556bdf959d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556bdf94c170 0x556bdf9597d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 --2- 172.21.15.7:0/3395228644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556bdf9468a0 0x556bdf959290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 >> 172.21.15.7:0/3395228644 conn(0x556bdf94e070 msgr2=0x556bdf952ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f69f2a00640 0 client.16995 ms_handle_reset on v2:172.21.15.186:6838/2448649165 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 shutdown_connections 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 1 -- 172.21.15.7:0/3395228644 wait complete. 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.238+0000 7f6a12d84740 20 client.16995 trim_cache size 0 max 16384 2024-04-20T21:43:45.245 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 msgr2=0x7f69ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 0x7f69ec0a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x556bdf947a30 tx=0x7f69f4009830 comp rx=0 tx=0).stop 2024-04-20T21:43:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 msgr2=0x7f6a0819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9b0 tx=0x7f6a0000ee70 comp rx=0 tx=0).stop 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 shutdown_connections 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a0810c360 0x7f6a0819a790 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a08103860 0x7f6a081a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f69ec0a3d20 0x7f69ec0a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 --2- 172.21.15.7:0/4251898975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a08102f40 0x7f6a081a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 >> 172.21.15.7:0/4251898975 conn(0x7f6a08077a10 msgr2=0x7f6a080fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 shutdown_connections 2024-04-20T21:43:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.239+0000 7f6a0ea00640 1 -- 172.21.15.7:0/4251898975 wait complete. 2024-04-20T21:43:45.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-20T21:43:45.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 -- 172.21.15.7:0/4089071926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8054109000 msgr2=0x7f8054115520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 --2- 172.21.15.7:0/4089071926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8054109000 0x7f8054115520 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f804c009cd0 tx=0x7f804c02c0e0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 -- 172.21.15.7:0/4089071926 shutdown_connections 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 --2- 172.21.15.7:0/4089071926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 0x7f8054108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 --2- 172.21.15.7:0/4089071926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8054109000 0x7f8054115520 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 --2- 172.21.15.7:0/4089071926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f805410a400 0x7f805410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 -- 172.21.15.7:0/4089071926 >> 172.21.15.7:0/4089071926 conn(0x7f8054077dd0 msgr2=0x7f8054106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 -- 172.21.15.7:0/4089071926 shutdown_connections 2024-04-20T21:43:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.472+0000 7f805a400640 1 -- 172.21.15.7:0/4089071926 wait complete. 2024-04-20T21:43:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.473+0000 7f805a400640 1 Processor -- start 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.473+0000 7f805a400640 1 -- start start 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 0x7f805419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 0x7f805419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80541189d0 con 0x7f805410a400 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8054118850 con 0x7f8054108680 2024-04-20T21:43:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f805a400640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f8054118b50 con 0x7f8054109000 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f8053e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f8052a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 0x7f805419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f8053e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39112/0 (socket says 172.21.15.7:39112) 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f8053e00640 1 -- 172.21.15.7:0/217913286 learned_addr learned my addr 172.21.15.7:0/217913286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.474+0000 7f8053400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 0x7f805419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 -- 172.21.15.7:0/217913286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 msgr2=0x7f805419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 0x7f805419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 -- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 msgr2=0x7f80541a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 -- 172.21.15.7:0/217913286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80541a3dc0 con 0x7f8054109000 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8053e00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f8052a00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 0x7f805419f310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9f0 tx=0x7f804000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f804be00640 1 -- 172.21.15.7:0/217913286 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000ccc0 con 0x7f8054109000 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f804be00640 1 -- 172.21.15.7:0/217913286 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8040010910 con 0x7f8054109000 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80541a40b0 con 0x7f8054109000 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f804be00640 1 -- 172.21.15.7:0/217913286 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040018960 con 0x7f8054109000 2024-04-20T21:43:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.475+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541ab940 con 0x7f8054109000 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 Processor -- start 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 -- start start 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f804be00640 1 -- 172.21.15.7:0/217913286 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f8040010ac0 con 0x7f8054109000 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b6a616050 con 0x563b6a6ba4b0 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b6a5bc490 con 0x563b6a6b18d0 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.479+0000 7f805d1df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563b6a612010 con 0x563b6a6b0fb0 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48662/0 (socket says 172.21.15.7:48662) 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37780/0 (socket says 172.21.15.7:37780) 2024-04-20T21:43:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053e00640 1 -- 172.21.15.7:0/2437432061 learned_addr learned my addr 172.21.15.7:0/2437432061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8052a00640 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 -- 172.21.15.7:0/2437432061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 msgr2=0x563b6a6b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 -- 172.21.15.7:0/2437432061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 msgr2=0x563b6a6bc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6bc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 -- 172.21.15.7:0/2437432061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6a5c4070 con 0x563b6a6b18d0 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.480+0000 7f8053400640 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f804400bbf0 tx=0x7f80440107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed04f2070b4b5048 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f804be00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 0x7f80380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f8048c00640 1 -- 172.21.15.7:0/2437432061 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044004030 con 0x563b6a6b18d0 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f8048c00640 1 -- 172.21.15.7:0/2437432061 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80440041d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f804be00640 1 -- 172.21.15.7:0/217913286 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f80400d0b70 con 0x7f8054109000 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f8048c00640 1 -- 172.21.15.7:0/2437432061 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80440116c0 con 0x563b6a6b18d0 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f8053e00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 0x7f80380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f805d1df740 1 -- 172.21.15.7:0/2437432061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 msgr2=0x563b6a6b9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.481+0000 7f805d1df740 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f804400bbf0 tx=0x7f80440107a0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 -- 172.21.15.7:0/2437432061 shutdown_connections 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b9f70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6bc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 --2- 172.21.15.7:0/2437432061 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 -- 172.21.15.7:0/2437432061 >> 172.21.15.7:0/2437432061 conn(0x563b6a6ab6e0 msgr2=0x563b6a6abac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 -- 172.21.15.7:0/2437432061 shutdown_connections 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 -- 172.21.15.7:0/2437432061 wait complete. 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f8053e00640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 0x7f80380a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f804c009a60 tx=0x7f804c005d80 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 -- start start 2024-04-20T21:43:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.482+0000 7f805d1df740 1 Processor -- start 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b6a6bd2a0 con 0x563b6a6b0fb0 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b6a5cf860 con 0x563b6a6b18d0 2024-04-20T21:43:45.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x563b6a6bd420 con 0x563b6a6ba4b0 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f805d1df740 10 client.0 _ll_register_callbacks cb 0x563b6a59dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f8053400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f8053400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39140/0 (socket says 172.21.15.7:39140) 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f8053400640 1 -- 172.21.15.7:0/3842813047 learned_addr learned my addr 172.21.15.7:0/3842813047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f8052a00640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.483+0000 7f8053e00640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 -- 172.21.15.7:0/3842813047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6ba4b0 msgr2=0x563b6a6b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 -- 172.21.15.7:0/3842813047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 msgr2=0x563b6a6b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 -- 172.21.15.7:0/3842813047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6a5d3950 con 0x563b6a6b18d0 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f8052a00640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b7aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80400ce280 tx=0x7f804009c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe5e4d2d48edb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f803d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80400a5740 con 0x563b6a6b18d0 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f80400ce500 con 0x563b6a6b18d0 2024-04-20T21:43:45.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.484+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80440041d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80400ce700 con 0x563b6a6b18d0 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 10 client.27245 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b6a5a1820 con 0x563b6a6b18d0 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80440041d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f80400988d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 10 client.27245 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 20 client.27245 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:45.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a6ca530 0x563b6a6cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 4 client.27245 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.485+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b6a623a70 con 0x563b6a6ca530 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.486+0000 7f8053400640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a6ca530 0x563b6a6cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.486+0000 7f8053400640 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a6ca530 0x563b6a6cc920 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308088e0cb336917 server_cookie=648309759bb98bcf in_seq=0 out_seq=0 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.486+0000 7f803d600640 10 client.27245 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f803d600640 1 -- 172.21.15.7:0/3842813047 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b6a623a70 con 0x563b6a6ca530 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f803d600640 10 client.27245 handle_command_reply: tid=0 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 client.27245 shutdown 2024-04-20T21:43:45.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 msgr2=0x563b6a6b7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b7aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80400ce280 tx=0x7f804009c780 comp rx=0 tx=0).stop 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 shutdown_connections 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b7aa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a6ca530 0x563b6a6cc920 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 --2- 172.21.15.7:0/3842813047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f803d600640 0 client.27245 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.487+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 >> 172.21.15.7:0/3842813047 conn(0x563b6a6ab6e0 msgr2=0x563b6a6ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.488+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 shutdown_connections 2024-04-20T21:43:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.488+0000 7f805d1df740 1 -- 172.21.15.7:0/3842813047 wait complete. 2024-04-20T21:43:45.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.488+0000 7f805d1df740 20 client.27245 trim_cache size 0 max 16384 2024-04-20T21:43:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.502+0000 7f805d1df740 1 Processor -- start 2024-04-20T21:43:45.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 -- start start 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6ba4b0 0x563b6a6d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b69f0 con 0x563b6a6b18d0 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b42b0 con 0x563b6a6ba4b0 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f805d1df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b2120 con 0x563b6a6b0fb0 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f8052a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f8052a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39144/0 (socket says 172.21.15.7:39144) 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f8052a00640 1 -- 172.21.15.7:0/2629540278 learned_addr learned my addr 172.21.15.7:0/2629540278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f8053400640 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.503+0000 7f8053e00640 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6ba4b0 0x563b6a6d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 -- 172.21.15.7:0/2629540278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 msgr2=0x563b6a6b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 -- 172.21.15.7:0/2629540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6ba4b0 msgr2=0x563b6a6d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6ba4b0 0x563b6a6d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 -- 172.21.15.7:0/2629540278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6a3283f0 con 0x563b6a6b18d0 2024-04-20T21:43:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f8052a00640 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f804000cf10 tx=0x7f804009c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4629a8c7996a2ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f803fe00640 1 -- 172.21.15.7:0/2629540278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804009c1e0 con 0x563b6a6b18d0 2024-04-20T21:43:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f803fe00640 1 -- 172.21.15.7:0/2629540278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804009ae80 con 0x563b6a6b18d0 2024-04-20T21:43:45.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.504+0000 7f803fe00640 1 -- 172.21.15.7:0/2629540278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040018d80 con 0x563b6a6b18d0 2024-04-20T21:43:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- 172.21.15.7:0/2629540278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 msgr2=0x563b6a6c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f804000cf10 tx=0x7f804009c5a0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- 172.21.15.7:0/2629540278 shutdown_connections 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6ba4b0 0x563b6a6d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6b18d0 0x563b6a6c6160 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 --2- 172.21.15.7:0/2629540278 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- 172.21.15.7:0/2629540278 >> 172.21.15.7:0/2629540278 conn(0x563b6a6ab6e0 msgr2=0x563b6a6aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- 172.21.15.7:0/2629540278 shutdown_connections 2024-04-20T21:43:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- 172.21.15.7:0/2629540278 wait complete. 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.505+0000 7f805d1df740 1 -- start start 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 Processor -- start 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b2770 con 0x563b6a6ba4b0 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b3be0 con 0x563b6a6b18d0 2024-04-20T21:43:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f80541b2e40 con 0x563b6a6b0fb0 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f805d1df740 10 client.0 _ll_register_callbacks cb 0x7f804009c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.506+0000 7f8053400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8053e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8053400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48684/0 (socket says 172.21.15.7:48684) 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8053400640 1 -- 172.21.15.7:0/3112509782 learned_addr learned my addr 172.21.15.7:0/3112509782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 -- 172.21.15.7:0/3112509782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 msgr2=0x563b6a6c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 -- 172.21.15.7:0/3112509782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 msgr2=0x563b6a6b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 -- 172.21.15.7:0/3112509782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6a3f27d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8053400640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8053e00640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f8052a00640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b40b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f804009c880 tx=0x7f804009c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712596d3710d8ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.507+0000 7f803d600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040010640 con 0x563b6a6b18d0 2024-04-20T21:43:45.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mon.2 v2:172.21.15.16:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f8040018b00 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f804009ae80 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8040018d00 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 10 client.27251 fetch_fsmap learned FSMap version 27 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563b6a3f27d0 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mon.2 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f804009ae80 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mon.2 v2:172.21.15.16:3300/0 5 ==== fsmap(e 27) ==== 8339+0+0 (secure 0 0 0) 0x7f8040087130 con 0x563b6a6b18d0 2024-04-20T21:43:45.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 10 client.27251 fetch_fsmap finished waiting for FSMap version 27 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 10 client.27251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 20 client.27251 populate_metadata read hostname 'smithi007' 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a730e50 0x563b6a733270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 4 client.27251 mds_command: new command op to 24575 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.508+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 --> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b6a61d8e0 con 0x563b6a730e50 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f8053400640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a730e50 0x563b6a733270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f8053400640 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a730e50 0x563b6a733270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863dd305f0dd66fa server_cookie=e2a28e42b1cce16 in_seq=0 out_seq=0 2024-04-20T21:43:45.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f803d600640 10 client.27251 ms_handle_connect on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f803d600640 1 -- 172.21.15.7:0/3112509782 <== mds.0 v2:172.21.15.16:6838/518118418 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b6a61d8e0 con 0x563b6a730e50 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f803d600640 10 client.27251 handle_command_reply: tid=0 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.509+0000 7f805d1df740 1 client.27251 shutdown 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 msgr2=0x563b6a6b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b40b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f804009c880 tx=0x7f804009c610 comp rx=0 tx=0).stop 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 shutdown_connections 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b6a6b18d0 0x563b6a6b40b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.16:6838/518118418,v1:172.21.15.16:6839/518118418] conn(0x563b6a730e50 0x563b6a733270 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b6a6ba4b0 0x563b6a6b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 --2- 172.21.15.7:0/3112509782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b6a6b0fb0 0x563b6a6c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f803d600640 0 client.27251 ms_handle_reset on v2:172.21.15.16:6838/518118418 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 >> 172.21.15.7:0/3112509782 conn(0x563b6a6ab6e0 msgr2=0x563b6a6a9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 shutdown_connections 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 1 -- 172.21.15.7:0/3112509782 wait complete. 2024-04-20T21:43:45.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.510+0000 7f805d1df740 20 client.27251 trim_cache size 0 max 16384 2024-04-20T21:43:45.518 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-04-20T21:43:45.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 msgr2=0x7f80380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 0x7f80380a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f804c009a60 tx=0x7f804c005d80 comp rx=0 tx=0).stop 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 msgr2=0x7f805419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 0x7f805419f310 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f804000e9f0 tx=0x7f804000eeb0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 shutdown_connections 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805410a400 0x7f80541a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8054109000 0x7f805419f310 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f80380a3d20 0x7f80380a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 --2- 172.21.15.7:0/217913286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8054108680 0x7f805419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 >> 172.21.15.7:0/217913286 conn(0x7f8054077dd0 msgr2=0x7f8054078e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 shutdown_connections 2024-04-20T21:43:45.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:45.511+0000 7f805a400640 1 -- 172.21.15.7:0/217913286 wait complete. 2024-04-20T21:43:45.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:45 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3079667439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:45.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:45 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3079667439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:45.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:45 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3079667439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:43:46.529 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-20T21:43:46.529 INFO:tasks.fwd_scrub:done joining 2024-04-20T21:43:46.530 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-20T21:43:46.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:43:46.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 -- 172.21.15.7:0/1287165732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 msgr2=0x7f1ff8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 --2- 172.21.15.7:0/1287165732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff8102570 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009be0 tx=0x7f1fe802bef0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 -- 172.21.15.7:0/1287165732 shutdown_connections 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 --2- 172.21.15.7:0/1287165732 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 --2- 172.21.15.7:0/1287165732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8102b40 0x7f1ff8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 --2- 172.21.15.7:0/1287165732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff8102570 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 -- 172.21.15.7:0/1287165732 >> 172.21.15.7:0/1287165732 conn(0x7f1ff80fda30 msgr2=0x7f1ff80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 -- 172.21.15.7:0/1287165732 shutdown_connections 2024-04-20T21:43:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.767+0000 7f1fff800640 1 -- 172.21.15.7:0/1287165732 wait complete. 2024-04-20T21:43:46.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.768+0000 7f1fff800640 1 Processor -- start 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.768+0000 7f1fff800640 1 -- start start 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff8112340 con 0x7f1ff819a800 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81121c0 con 0x7f1ff8102190 2024-04-20T21:43:46.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fff800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff81124c0 con 0x7f1ff8107260 2024-04-20T21:43:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1ffd800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fefe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1ffce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fefe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48698/0 (socket says 172.21.15.7:48698) 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1ffd800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39182/0 (socket says 172.21.15.7:39182) 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.769+0000 7f1fefe00640 1 -- 172.21.15.7:0/145580672 learned_addr learned my addr 172.21.15.7:0/145580672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 -- 172.21.15.7:0/145580672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 msgr2=0x7f1ff81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:46.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff81a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 -- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 msgr2=0x7f1ff819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 -- 172.21.15.7:0/145580672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff819b390 con 0x7f1ff8107260 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1ffd800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fefe00640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e9a0 tx=0x7f1fe000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000cd80 con 0x7f1ff8107260 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe00035f0 con 0x7f1ff8107260 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.770+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff819b680 con 0x7f1ff8107260 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.771+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0010670 con 0x7f1ff8107260 2024-04-20T21:43:46.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.771+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff81a78b0 con 0x7f1ff8107260 2024-04-20T21:43:46.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.772+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff8103660 con 0x7f1ff8107260 2024-04-20T21:43:46.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.774+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1fe00037a0 con 0x7f1ff8107260 2024-04-20T21:43:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.775+0000 7f1ff6000640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 0x7f1fdc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.775+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f1fe00d0a40 con 0x7f1ff8107260 2024-04-20T21:43:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.775+0000 7f1ffce00640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 0x7f1fdc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:46.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.776+0000 7f1ffce00640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 0x7f1fdc0a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8004af0 tx=0x7f1fe80333e0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:46.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.779+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f1ff819b680 con 0x7f1ff8107260 2024-04-20T21:43:46.845 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:46 smithi016 ceph-mon[21409]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:46.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:46 smithi186 ceph-mon[22157]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:46.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.986+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ff8102b40 con 0x7f1ff8107260 2024-04-20T21:43:46.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.988+0000 7f1ff6000640 1 -- 172.21.15.7:0/145580672 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+22949 (secure 0 0 0) 0x7f1fe009d0a0 con 0x7f1ff8107260 2024-04-20T21:43:46.988 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:43:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:43:23.695002+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:43:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 msgr2=0x7f1fdc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 0x7f1fdc0a6260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8004af0 tx=0x7f1fe80333e0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 msgr2=0x7f1ff81a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:46.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e9a0 tx=0x7f1fe000ee60 comp rx=0 tx=0).stop 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 shutdown_connections 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff819a800 0x7f1ff819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ff8107260 0x7f1ff81a0bd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f1fdc0a3da0 0x7f1fdc0a6260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 --2- 172.21.15.7:0/145580672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff8102190 0x7f1ff81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 >> 172.21.15.7:0/145580672 conn(0x7f1ff80fda30 msgr2=0x7f1ff80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.991+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 shutdown_connections 2024-04-20T21:43:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:46.992+0000 7f1fff800640 1 -- 172.21.15.7:0/145580672 wait complete. 2024-04-20T21:43:47.006 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:46 smithi007 ceph-mon[16867]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-20T21:43:47.010 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:43:47.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:43:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 -- 172.21.15.7:0/2590489679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750103860 msgr2=0x7fe75010bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2590489679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750103860 0x7fe75010bd40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009900 tx=0x7fe73c02bcd0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 -- 172.21.15.7:0/2590489679 shutdown_connections 2024-04-20T21:43:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2590489679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe750103860 0x7fe75010bd40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2590489679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe75010c310 0x7fe75010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2590489679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 0x7fe750103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 -- 172.21.15.7:0/2590489679 >> 172.21.15.7:0/2590489679 conn(0x7fe750077a10 msgr2=0x7fe7500fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 -- 172.21.15.7:0/2590489679 shutdown_connections 2024-04-20T21:43:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.230+0000 7fe74fe00640 1 -- 172.21.15.7:0/2590489679 wait complete. 2024-04-20T21:43:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.231+0000 7fe74fe00640 1 Processor -- start 2024-04-20T21:43:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.231+0000 7fe74fe00640 1 -- start start 2024-04-20T21:43:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 0x7fe750198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 0x7fe75019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe750112140 con 0x7fe750103860 2024-04-20T21:43:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe750111fc0 con 0x7fe75010c310 2024-04-20T21:43:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74fe00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fe7501122c0 con 0x7fe750102f40 2024-04-20T21:43:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39194/0 (socket says 172.21.15.7:39194) 2024-04-20T21:43:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 -- 172.21.15.7:0/2742094468 learned_addr learned my addr 172.21.15.7:0/2742094468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74f400640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 0x7fe75019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74ea00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 0x7fe750198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 -- 172.21.15.7:0/2742094468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 msgr2=0x7fe750198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 0x7fe750198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 -- 172.21.15.7:0/2742094468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 msgr2=0x7fe75019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 0x7fe75019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74e000640 1 -- 172.21.15.7:0/2742094468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75019d5c0 con 0x7fe750103860 2024-04-20T21:43:47.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.232+0000 7fe74f400640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 0x7fe75019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe74e000640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c042020 con 0x7fe750103860 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73c0054e0 con 0x7fe750103860 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75019d850 con 0x7fe750103860 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c003af0 con 0x7fe750103860 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.233+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7501a5140 con 0x7fe750103860 2024-04-20T21:43:47.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.235+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75004fcf0 con 0x7fe750103860 2024-04-20T21:43:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.237+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fe73c045020 con 0x7fe750103860 2024-04-20T21:43:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.238+0000 7fe747400640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 0x7fe7340a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.238+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fe73c0f22e0 con 0x7fe750103860 2024-04-20T21:43:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.238+0000 7fe74ea00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 0x7fe7340a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.239+0000 7fe74ea00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 0x7fe7340a61d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe740009bd0 tx=0x7fe740005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.242+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fe73c04c020 con 0x7fe750103860 2024-04-20T21:43:47.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.458+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe750107460 con 0x7fe750103860 2024-04-20T21:43:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.459+0000 7fe747400640 1 -- 172.21.15.7:0/2742094468 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fe73c0bea50 con 0x7fe750103860 2024-04-20T21:43:47.457 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:43:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 msgr2=0x7fe7340a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 0x7fe7340a61d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe740009bd0 tx=0x7fe740005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 msgr2=0x7fe750198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe73c000c00 tx=0x7fe73c004360 comp rx=0 tx=0).stop 2024-04-20T21:43:47.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 shutdown_connections 2024-04-20T21:43:47.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fe7340a3d10 0x7fe7340a61d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75010c310 0x7fe75019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe750103860 0x7fe750198b60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 --2- 172.21.15.7:0/2742094468 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe750102f40 0x7fe750198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.462+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 >> 172.21.15.7:0/2742094468 conn(0x7fe750077a10 msgr2=0x7fe7500fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.463+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 shutdown_connections 2024-04-20T21:43:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.463+0000 7fe74fe00640 1 -- 172.21.15.7:0/2742094468 wait complete. 2024-04-20T21:43:47.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:43:47.600 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:47 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/145580672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 -- 172.21.15.7:0/1520120357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1278107260 msgr2=0x7f127810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 --2- 172.21.15.7:0/1520120357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1278107260 0x7f127810ee30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1270009ca0 tx=0x7f127002c100 comp rx=0 tx=0).stop 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 -- 172.21.15.7:0/1520120357 shutdown_connections 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 --2- 172.21.15.7:0/1520120357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1278107260 0x7f127810ee30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 --2- 172.21.15.7:0/1520120357 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102b40 0x7f1278106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 --2- 172.21.15.7:0/1520120357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278102190 0x7f1278102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 -- 172.21.15.7:0/1520120357 >> 172.21.15.7:0/1520120357 conn(0x7f12780fda10 msgr2=0x7f12780ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 -- 172.21.15.7:0/1520120357 shutdown_connections 2024-04-20T21:43:47.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.668+0000 7f127fc00640 1 -- 172.21.15.7:0/1520120357 wait complete. 2024-04-20T21:43:47.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 Processor -- start 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 -- start start 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102190 0x7f127819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278107260 0x7f127819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1278112300 con 0x7f1278107260 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1278112180 con 0x7f12781985c0 2024-04-20T21:43:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127fc00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f1278112480 con 0x7f1278102190 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.669+0000 7f127dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:37856/0 (socket says 172.21.15.7:37856) 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127d200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102190 0x7f127819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 -- 172.21.15.7:0/1176261463 learned_addr learned my addr 172.21.15.7:0/1176261463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f1277e00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278107260 0x7f127819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 -- 172.21.15.7:0/1176261463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102190 msgr2=0x7f127819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102190 0x7f127819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 -- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278107260 msgr2=0x7f127819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278107260 0x7f127819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 -- 172.21.15.7:0/1176261463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12781990e0 con 0x7f12781985c0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f127dc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1270037040 tx=0x7f1270032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.670+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270039040 con 0x7f12781985c0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.671+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1270044070 con 0x7f12781985c0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.671+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12700042a0 con 0x7f12781985c0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.671+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1278199370 con 0x7f12781985c0 2024-04-20T21:43:47.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.671+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12781a55c0 con 0x7f12781985c0 2024-04-20T21:43:47.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.672+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1278069260 con 0x7f12781985c0 2024-04-20T21:43:47.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.674+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f1270003970 con 0x7f12781985c0 2024-04-20T21:43:47.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.675+0000 7f1275600640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 0x7f12600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:47.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.675+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f12700f2b40 con 0x7f12781985c0 2024-04-20T21:43:47.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.675+0000 7f127d200640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 0x7f12600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:47.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.676+0000 7f127d200640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 0x7f12600a6260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f126c0095f0 tx=0x7f126c005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.679+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f12700bf2b0 con 0x7f12781985c0 2024-04-20T21:43:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.891+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1278102b40 con 0x7f12781985c0 2024-04-20T21:43:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.892+0000 7f1275600640 1 -- 172.21.15.7:0/1176261463 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f12700bebb0 con 0x7f12781985c0 2024-04-20T21:43:47.891 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 msgr2=0x7f12600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 0x7f12600a6260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f126c0095f0 tx=0x7f126c005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 msgr2=0x7f1278198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1270037040 tx=0x7f1270032f90 comp rx=0 tx=0).stop 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 shutdown_connections 2024-04-20T21:43:47.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f12600a3da0 0x7f12600a6260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12781985c0 0x7f1278198a70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1278107260 0x7f127819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 --2- 172.21.15.7:0/1176261463 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1278102190 0x7f127819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 >> 172.21.15.7:0/1176261463 conn(0x7f12780fda10 msgr2=0x7f12780ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 shutdown_connections 2024-04-20T21:43:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:47.895+0000 7f127fc00640 1 -- 172.21.15.7:0/1176261463 wait complete. 2024-04-20T21:43:47.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:47 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.911 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:47 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2742094468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:43:47.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:43:47.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:47 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/145580672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:47 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:47 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2742094468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:43:47.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:47 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/145580672' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:47 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-20T21:43:47.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:47 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2742094468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:43:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 -- 172.21.15.7:0/3003177813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c107240 msgr2=0x7f540c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 --2- 172.21.15.7:0/3003177813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c107240 0x7f540c10ee10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5404009ca0 tx=0x7f540402c100 comp rx=0 tx=0).stop 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 -- 172.21.15.7:0/3003177813 shutdown_connections 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 --2- 172.21.15.7:0/3003177813 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c107240 0x7f540c10ee10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 --2- 172.21.15.7:0/3003177813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069f30 0x7f540c106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 --2- 172.21.15.7:0/3003177813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c069580 0x7f540c069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 -- 172.21.15.7:0/3003177813 >> 172.21.15.7:0/3003177813 conn(0x7f540c077fb0 msgr2=0x7f540c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 -- 172.21.15.7:0/3003177813 shutdown_connections 2024-04-20T21:43:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.114+0000 7f5412200640 1 -- 172.21.15.7:0/3003177813 wait complete. 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.115+0000 7f5412200640 1 Processor -- start 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.115+0000 7f5412200640 1 -- start start 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 0x7f540c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 0x7f540c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1122e0 con 0x7f540c069580 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f540c112160 con 0x7f540c107240 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f5412200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f540c112460 con 0x7f540c19a820 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f540b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f540aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 0x7f540c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f540b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39248/0 (socket says 172.21.15.7:39248) 2024-04-20T21:43:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f540b400640 1 -- 172.21.15.7:0/2101138719 learned_addr learned my addr 172.21.15.7:0/2101138719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.116+0000 7f540be00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 0x7f540c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 -- 172.21.15.7:0/2101138719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 msgr2=0x7f540c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 0x7f540c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 -- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 msgr2=0x7f540c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 -- 172.21.15.7:0/2101138719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c19b3b0 con 0x7f540c107240 2024-04-20T21:43:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540b400640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540be00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 0x7f540c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f540aa00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 0x7f540c1a0bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9c0 tx=0x7f53f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f400ccc0 con 0x7f540c107240 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f4010910 con 0x7f540c107240 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c19b6a0 con 0x7f540c107240 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f40178e0 con 0x7f540c107240 2024-04-20T21:43:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.117+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a7880 con 0x7f540c107240 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.119+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c103640 con 0x7f540c107240 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.121+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f53f4010ac0 con 0x7f540c107240 2024-04-20T21:43:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.122+0000 7f5403e00640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 0x7f53f00a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.122+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f53f40d0cd0 con 0x7f540c107240 2024-04-20T21:43:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.122+0000 7f540b400640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 0x7f53f00a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.123+0000 7f540b400640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 0x7f53f00a6220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53fc011550 tx=0x7f53fc005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.131+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f540c19b6a0 con 0x7f540c107240 2024-04-20T21:43:48.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.352+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f540c068e10 con 0x7f540c107240 2024-04-20T21:43:48.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.353+0000 7f5403e00640 1 -- 172.21.15.7:0/2101138719 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f53f409d330 con 0x7f540c107240 2024-04-20T21:43:48.351 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:43:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.355+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 msgr2=0x7f53f00a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.355+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 0x7f53f00a6220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f53fc011550 tx=0x7f53fc005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.355+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 msgr2=0x7f540c1a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.355+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 0x7f540c1a0bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f53f400e9c0 tx=0x7f53f400ee80 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 shutdown_connections 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f53f00a3d60 0x7f53f00a6220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f540c107240 0x7f540c1a0bf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f540c19a820 0x7f540c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 --2- 172.21.15.7:0/2101138719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f540c069580 0x7f540c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 >> 172.21.15.7:0/2101138719 conn(0x7f540c077fb0 msgr2=0x7f540c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 shutdown_connections 2024-04-20T21:43:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.356+0000 7f5412200640 1 -- 172.21.15.7:0/2101138719 wait complete. 2024-04-20T21:43:48.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-20T21:43:48.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 -- 172.21.15.7:0/4119350055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18102f40 msgr2=0x7fcd18103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 --2- 172.21.15.7:0/4119350055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18102f40 0x7fcd18103320 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd0802c180 comp rx=0 tx=0).stop 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 -- 172.21.15.7:0/4119350055 shutdown_connections 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 --2- 172.21.15.7:0/4119350055 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd1810c310 0x7fcd1810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 --2- 172.21.15.7:0/4119350055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd18103860 0x7fcd1810bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 --2- 172.21.15.7:0/4119350055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18102f40 0x7fcd18103320 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.588+0000 7fcd17e00640 1 -- 172.21.15.7:0/4119350055 >> 172.21.15.7:0/4119350055 conn(0x7fcd18077a10 msgr2=0x7fcd180fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.589+0000 7fcd17e00640 1 -- 172.21.15.7:0/4119350055 shutdown_connections 2024-04-20T21:43:48.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.589+0000 7fcd17e00640 1 -- 172.21.15.7:0/4119350055 wait complete. 2024-04-20T21:43:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.590+0000 7fcd17e00640 1 Processor -- start 2024-04-20T21:43:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.590+0000 7fcd17e00640 1 -- start start 2024-04-20T21:43:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18103860 0x7fcd1819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 0x7fcd1819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd181120f0 con 0x7fcd1810c310 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111f70 con 0x7fcd18103860 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18112270 con 0x7fcd18102f40 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:48752/0 (socket says 172.21.15.7:48752) 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd16a00640 1 -- 172.21.15.7:0/2883663574 learned_addr learned my addr 172.21.15.7:0/2883663574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd0e000640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18103860 0x7fcd1819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.591+0000 7fcd17400640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 0x7fcd1819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 -- 172.21.15.7:0/2883663574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18103860 msgr2=0x7fcd1819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18103860 0x7fcd1819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 -- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 msgr2=0x7fcd1819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 0x7fcd1819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 -- 172.21.15.7:0/2883663574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1819f740 con 0x7fcd18102f40 2024-04-20T21:43:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd17400640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 0x7fcd1819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:43:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd16a00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005e30 tx=0x7fcd08035d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08039040 con 0x7fcd18102f40 2024-04-20T21:43:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd08044070 con 0x7fcd18102f40 2024-04-20T21:43:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080328d0 con 0x7fcd18102f40 2024-04-20T21:43:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.592+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1819f9d0 con 0x7fcd18102f40 2024-04-20T21:43:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.593+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd181a72c0 con 0x7fcd18102f40 2024-04-20T21:43:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.594+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd18069b00 con 0x7fcd18102f40 2024-04-20T21:43:48.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.597+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fcd08032a70 con 0x7fcd18102f40 2024-04-20T21:43:48.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.599+0000 7fcd0fe00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 0x7fccfc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:43:48.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.599+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fcd080f2b90 con 0x7fcd18102f40 2024-04-20T21:43:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.599+0000 7fcd0e000640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 0x7fccfc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:43:48.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.600+0000 7fcd0e000640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 0x7fccfc0a6300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd1810e310 tx=0x7fcd00005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:43:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.603+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fcd080bf300 con 0x7fcd18102f40 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1176261463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:48.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:48 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2101138719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:43:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.872+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fcd181a7490 con 0x7fcd18102f40 2024-04-20T21:43:48.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.882+0000 7fcd0fe00640 1 -- 172.21.15.7:0/2883663574 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746049 (secure 0 0 0) 0x7fcd080bec00 con 0x7fcd18102f40 2024-04-20T21:43:48.894 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 27, 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "active_gid": 25208, 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "active_name": "x", 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addrs": { 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:48.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2899881467 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-04-20T21:43:48.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2899881467 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "active_addr": "172.21.15.7:6801/2899881467", 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "active_change": "2024-04-20T21:33:20.988778+0000", 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "active_mgr_features": 4540701547738038271, 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "available": true, 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: "standbys": [ 2024-04-20T21:43:48.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 25205, 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "y", 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-04-20T21:43:48.898 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:48.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:48.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.901 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.902 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.905 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.909 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-04-20T21:43:48.910 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.911 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-04-20T21:43:48.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-04-20T21:43:48.913 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.914 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.915 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-04-20T21:43:48.916 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-04-20T21:43:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.919 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:48.920 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.921 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-04-20T21:43:48.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.927 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:48.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-04-20T21:43:48.933 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-04-20T21:43:48.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:43:48.935 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.936 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:48.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:48.943 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-04-20T21:43:48.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.948 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-04-20T21:43:48.949 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-04-20T21:43:48.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:48.954 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:43:48.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:48.957 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.958 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-04-20T21:43:48.959 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:43:48.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-04-20T21:43:48.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:43:48.965 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-04-20T21:43:48.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:48.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.974 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.975 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.978 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:43:48.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-04-20T21:43:48.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-04-20T21:43:48.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.990 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-04-20T21:43:48.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-04-20T21:43:48.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.997 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:48.998 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:48.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.002 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:43:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:43:49.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-04-20T21:43:49.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:43:49.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-04-20T21:43:49.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.017 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.019 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.020 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.021 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.025 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.029 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.032 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.033 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-04-20T21:43:49.034 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-04-20T21:43:49.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.039 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.040 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-04-20T21:43:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.043 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.049 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.050 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.051 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.052 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.053 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-04-20T21:43:49.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:43:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.056 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.057 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.058 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.059 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.061 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:43:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:43:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:43:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.094 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.095 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:43:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:43:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:43:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:43:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "[]", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:43:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:43:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:43:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:43:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:43:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-04-20T21:43:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:43:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-04-20T21:43:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-04-20T21:43:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-04-20T21:43:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:43:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-04-20T21:43:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:43:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-04-20T21:43:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-04-20T21:43:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-04-20T21:43:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-04-20T21:43:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-04-20T21:43:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:43:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-04-20T21:43:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-04-20T21:43:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:43:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-04-20T21:43:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-04-20T21:43:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-04-20T21:43:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.356 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.359 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.361 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.363 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.364 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.365 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.367 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.374 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.375 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.376 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.378 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.382 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-04-20T21:43:49.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-04-20T21:43:49.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.391 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:43:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-04-20T21:43:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-04-20T21:43:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-04-20T21:43:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-04-20T21:43:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.499 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.502 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-04-20T21:43:49.506 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.509 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.510 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.511 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.515 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.517 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-04-20T21:43:49.518 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-04-20T21:43:49.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-04-20T21:43:49.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.530 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:49.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-04-20T21:43:49.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-04-20T21:43:49.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-04-20T21:43:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.541 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.545 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.546 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.548 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.554 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.557 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.558 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.560 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-04-20T21:43:49.561 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.563 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.564 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.565 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.567 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-04-20T21:43:49.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:43:49.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:43:49.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:43:49.576 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-04-20T21:43:49.577 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-04-20T21:43:49.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.579 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.580 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.585 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.586 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.587 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-04-20T21:43:49.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: "gid": 25210, 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "z", 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_features": 4540701547738038271, 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.596 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.598 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.599 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.600 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.602 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.603 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.605 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-04-20T21:43:49.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:43:49.607 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.609 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.615 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.616 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-04-20T21:43:49.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-04-20T21:43:49.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.620 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-04-20T21:43:49.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:49.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-04-20T21:43:49.624 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.626 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.627 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:43:49.628 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.629 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-04-20T21:43:49.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:43:49.631 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-04-20T21:43:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:43:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-04-20T21:43:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:43:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-04-20T21:43:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-04-20T21:43:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:43:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-04-20T21:43:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-04-20T21:43:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-04-20T21:43:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-04-20T21:43:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:43:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.694 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-04-20T21:43:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-04-20T21:43:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-04-20T21:43:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-04-20T21:43:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:43:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:43:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:43:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-04-20T21:43:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-04-20T21:43:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-04-20T21:43:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-04-20T21:43:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:43:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-04-20T21:43:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:43:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:43:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:43:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:43:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "[]", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-04-20T21:43:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:43:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:43:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:43:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:43:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:43:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:43:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.835 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-04-20T21:43:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-04-20T21:43:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.849 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.852 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.853 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.855 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.859 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.865 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:49.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.871 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.872 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.873 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.874 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.875 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.876 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-04-20T21:43:49.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.881 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.883 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-04-20T21:43:49.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.887 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:43:49.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.890 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.891 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.893 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.896 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.899 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:49.900 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-04-20T21:43:49.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.902 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.904 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:43:49.905 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-04-20T21:43:49.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.911 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.917 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-04-20T21:43:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:43:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-04-20T21:43:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:43:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-04-20T21:43:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-04-20T21:43:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:43:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-04-20T21:43:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:43:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.011 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.012 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.013 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.015 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-04-20T21:43:50.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-04-20T21:43:50.017 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.020 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.021 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.022 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.024 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.025 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.026 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.027 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.029 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.031 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.033 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.036 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-04-20T21:43:50.037 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.038 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.039 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-04-20T21:43:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-04-20T21:43:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-04-20T21:43:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-04-20T21:43:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-04-20T21:43:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:43:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-04-20T21:43:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-04-20T21:43:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-04-20T21:43:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-04-20T21:43:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-04-20T21:43:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-04-20T21:43:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-04-20T21:43:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.154 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.155 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.157 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.158 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.159 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.160 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.162 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.163 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.164 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.165 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-04-20T21:43:50.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.167 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:50.170 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-04-20T21:43:50.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.172 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.173 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.174 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.176 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.176 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-04-20T21:43:50.177 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.178 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.180 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.182 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.183 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.184 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.186 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.187 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.188 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-04-20T21:43:50.189 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.190 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.191 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.192 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.193 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.194 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.196 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.197 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.198 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.199 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.200 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.202 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.204 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.206 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.207 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:43:50.208 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.210 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.211 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.212 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.214 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.215 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.216 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.218 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.219 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.220 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-04-20T21:43:50.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.223 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.224 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.225 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.228 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.229 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:43:50.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:50.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.233 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.235 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-04-20T21:43:50.236 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.237 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.238 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-04-20T21:43:50.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.241 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.247 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.248 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.249 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.250 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-04-20T21:43:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.254 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.255 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.258 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.259 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.260 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.262 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.263 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.267 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.268 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.269 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-04-20T21:43:50.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.271 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.272 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.275 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.278 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.279 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.280 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.281 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.282 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.283 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.284 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.286 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.287 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.288 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.289 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.290 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.292 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.293 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: "modules": [ 2024-04-20T21:43:50.295 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "iostat", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nfs", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "restful", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_schedule" 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "available_modules": [ 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "alerts", 2024-04-20T21:43:50.296 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.297 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How frequently to reexamine health status", 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.299 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.301 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_destination": { 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_destination", 2024-04-20T21:43:50.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email address to send alerts to", 2024-04-20T21:43:50.307 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_from_name": { 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_from_name", 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.308 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "Ceph", 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Email From: name", 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.309 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_host": { 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_host", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.310 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP server", 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_password": { 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_password", 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.311 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Password to authenticate with", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_port": { 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_port", 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.313 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "465", 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP port", 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_sender": { 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_sender", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.315 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SMTP envelope sender", 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_ssl": { 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_ssl", 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.316 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "smtp_user": { 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "smtp_user", 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.318 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "User to authenticate as", 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "balancer", 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "active": { 2024-04-20T21:43:50.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "active", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically balance PGs across cluster", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_time": { 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_time", 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.322 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0000", 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "beginning time of day to automatically balance", 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "begin_weekday": { 2024-04-20T21:43:50.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "begin_weekday", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.324 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_max_iterations": { 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_max_iterations", 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "25", 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "250", 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_metrics": { 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_metrics", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "pgs,objects,bytes", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.327 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_compat_step": { 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crush_compat_step", 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.5", 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0.001", 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "0.999", 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "aggressiveness of optimization", 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.329 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "end_time": { 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_time", 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2359", 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.330 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ending time of day to automatically balance", 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "end_weekday": { 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "end_weekday", 2024-04-20T21:43:50.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "0", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "6", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-20T21:43:50.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.333 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.334 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.335 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.336 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.337 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.338 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min_score": { 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_score", 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.341 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:50.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "upmap", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-compat", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "none", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "read", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap-read" 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Balancer mode", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_ids": { 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_ids", 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.345 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.347 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.348 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_deviation": { 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_deviation", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.349 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "upmap_max_optimizations": { 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "upmap_max_optimizations", 2024-04-20T21:43:50.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-20T21:43:50.351 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm", 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_down_multiplier": { 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_down_multiplier", 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.353 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_refresh_rate": { 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_refresh_rate", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "20", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.355 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "agent_starting_port": { 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "agent_starting_port", 2024-04-20T21:43:50.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4721", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_ptrace": { 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_ptrace", 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.358 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.359 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_interval": { 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_interval", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.360 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "autotune_memory_target_ratio": { 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "autotune_memory_target_ratio", 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0.7", 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.362 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_log_destination": { 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cephadm_log_destination", 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.363 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "file", 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "file,syslog", 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "syslog" 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.364 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "cgroups_split": { 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cgroups_split", 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.365 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "config_checks_enabled": { 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_checks_enabled", 2024-04-20T21:43:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "config_dashboard": { 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_dashboard", 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_alertmanager": { 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_alertmanager", 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_base": { 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_base", 2024-04-20T21:43:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Container image name, without the tag", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_elasticsearch": { 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_elasticsearch", 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "elasticsearch container image", 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_grafana": { 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_grafana", 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_haproxy": { 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_haproxy", 2024-04-20T21:43:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "HAproxy container image", 2024-04-20T21:43:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_agent": { 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_agent", 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger agent container image", 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_collector": { 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_collector", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger collector container image", 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_jaeger_query": { 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_jaeger_query", 2024-04-20T21:43:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Jaeger query container image", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_keepalived": { 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_keepalived", 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-20T21:43:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Keepalived container image", 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_loki": { 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_loki", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Loki container image", 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_node_exporter": { 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_node_exporter", 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_nvmeof": { 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_nvmeof", 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Nvme-of container image", 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_prometheus": { 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_prometheus", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Prometheus container image", 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_promtail": { 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_promtail", 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Promtail container image", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_samba": { 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_samba", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Samba/SMB container image", 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image_snmp_gateway": { 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_image_snmp_gateway", 2024-04-20T21:43:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "SNMP Gateway container image", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "container_init": { 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "container_init", 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Run podman/docker with `--init`", 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "daemon_cache_timeout": { 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "daemon_cache_timeout", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "default_cephadm_command_timeout": { 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_cephadm_command_timeout", 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "900", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_registry": { 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "default_registry", 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "docker.io", 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "device_cache_timeout": { 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_cache_timeout", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1800", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache device inventory", 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "device_enhanced_scan": { 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_enhanced_scan", 2024-04-20T21:43:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-20T21:43:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "facts_cache_timeout": { 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "facts_cache_timeout", 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "seconds to cache host facts data", 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "grafana_dashboards_path": { 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "grafana_dashboards_path", 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-20T21:43:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "host_check_interval": { 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "host_check_interval", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to perform a host check", 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "hw_monitoring": { 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hw_monitoring", 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "inventory_list_all": { 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "inventory_list_all", 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "log_refresh_metadata": { 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_refresh_metadata", 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-20T21:43:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-20T21:43:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "*", 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max_count_per_host": { 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_count_per_host", 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of daemons per service per host", 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd_draining_count": { 2024-04-20T21:43:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_osd_draining_count", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "migration_current": { 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "migration_current", 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "internal - do not modify", 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": { 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mode", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "root", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm-package", 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "root" 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "mode for remote execution of cephadm", 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "oob_default_addr": { 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "oob_default_addr", 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "169.254.1.1", 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "prometheus_alerts_path": { 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus_alerts_path", 2024-04-20T21:43:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_insecure": { 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_insecure", 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_password": { 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_password", 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_url": { 2024-04-20T21:43:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_url", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "registry_username": { 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "registry_username", 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "secure_monitoring_stack": { 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "secure_monitoring_stack", 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "service_discovery_port": { 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "service_discovery_port", 2024-04-20T21:43:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8765", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "cephadm service discovery port", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ssh_config_file": { 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssh_config_file", 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "use_agent": { 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_agent", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "use_repo_digest": { 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "use_repo_digest", 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_failed_host_check": { 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_failed_host_check", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if the host check fails", 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_daemons": { 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_daemons", 2024-04-20T21:43:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_on_stray_hosts": { 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_on_stray_hosts", 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crash", 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "retain_interval": { 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retain_interval", 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "31536000", 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_recent_interval": { 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_recent_interval", 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1209600", 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dashboard", 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_HOST": { 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-20T21:43:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_ENABLED": { 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_ENABLED", 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "ENABLE_BROWSABLE_API": { 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-20T21:43:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-20T21:43:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_PASSWORD": { 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-20T21:43:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_URL": { 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_URL", 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_API_USERNAME": { 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "[]", 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "MULTICLUSTER_CONFIG": { 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "{}", 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_HOST": { 2024-04-20T21:43:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-20T21:43:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_ENABLED": { 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.527 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.528 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.529 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8", 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "45", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.532 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ACCESS_KEY": { 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.533 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "admin", 2024-04-20T21:43:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.536 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SECRET_KEY": { 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.537 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "RGW_API_SSL_VERIFY": { 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-20T21:43:50.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "UNSAFE_TLS_v1_2": { 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.540 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.542 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.543 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.544 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:50.545 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "cross_origin_url": { 2024-04-20T21:43:50.546 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cross_origin_url", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.547 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "crt_file": { 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "crt_file", 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.548 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "debug": { 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "debug", 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.550 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Enable/disable debug options", 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.551 INFO:teuthology.orchestra.run.smithi007.stdout: "jwt_token_ttl": { 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "jwt_token_ttl", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "28800", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:50.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.554 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.555 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.556 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.558 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.559 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.560 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.562 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "motd": { 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "motd", 2024-04-20T21:43:50.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "The message of the day", 2024-04-20T21:43:50.564 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect_resolve_ip_addr": { 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "redirect_resolve_ip_addr", 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.565 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.566 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:50.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8080", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.569 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl_server_port": { 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl_server_port", 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8443", 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.572 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:50.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "redirect", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.574 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "redirect" 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:50.575 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:50.576 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "url_prefix": { 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url_prefix", 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.577 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.579 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_monitoring": { 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_monitoring", 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.580 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "monitor device health metrics", 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.581 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.582 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.584 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.585 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.586 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.588 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.589 INFO:teuthology.orchestra.run.smithi007.stdout: "mark_out_threshold": { 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mark_out_threshold", 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "2419200", 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.590 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": { 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pool_name", 2024-04-20T21:43:50.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "device_health_metrics", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "retention_period": { 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "retention_period", 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.593 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15552000", 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long to retain device health metrics", 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_frequency": { 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_frequency", 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.595 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.596 INFO:teuthology.orchestra.run.smithi007.stdout: "self_heal": { 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "self_heal", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how frequently to wake up and check device health", 2024-04-20T21:43:50.599 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "warn_threshold": { 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "warn_threshold", 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.600 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7257600", 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "diskprediction_local", 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.602 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.603 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.604 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.606 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.608 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.609 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.610 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "predict_interval": { 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predict_interval", 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "86400", 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "predictor_model": { 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "predictor_model", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "prophetstor", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.613 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "600", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "influx", 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": false, 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "influxdb python module not found", 2024-04-20T21:43:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "batch_size": { 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "batch_size", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5000", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "database": { 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "database", 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.618 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "ceph", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.619 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": { 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "hostname", 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.620 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server hostname", 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "30", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "5", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.622 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.623 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.625 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.626 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.627 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.628 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.629 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "password": { 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "password", 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "password of InfluxDB server user", 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.632 INFO:teuthology.orchestra.run.smithi007.stdout: "port": { 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "port", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "8086", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.633 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "InfluxDB server port", 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ssl": { 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ssl", 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.634 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "false", 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:50.635 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "threads": { 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threads", 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "32", 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.637 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "username": { 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "username", 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.638 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "username of InfluxDB server user", 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "verify_ssl": { 2024-04-20T21:43:50.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "verify_ssl", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "true", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.640 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "insights", 2024-04-20T21:43:50.641 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.642 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.643 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.644 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.645 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.647 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.649 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "iostat", 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.650 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.651 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.652 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.654 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.655 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.658 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "k8sevents", 2024-04-20T21:43:50.659 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_event_retention_days": { 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "ceph_event_retention_days", 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.660 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "7", 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "config_check_secs": { 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "config_check_secs", 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "10", 2024-04-20T21:43:50.662 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.664 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.666 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.669 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.671 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "localpool", 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.672 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "failure_domain": { 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "failure_domain", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "host", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.673 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "failure domain for any created local pool", 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.675 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.676 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.678 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.679 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.681 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.682 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": { 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "min_size", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.683 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default min_size for any created local pool", 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "num_rep": { 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "num_rep", 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3", 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default replica count for any created local pool", 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": { 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_num", 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "128", 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.686 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "default pg_num for any created local pool", 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "prefix": { 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prefix", 2024-04-20T21:43:50.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "name prefix for any created local pool", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "subtree": { 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "subtree", 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "rack", 2024-04-20T21:43:50.689 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mds_autoscaler", 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.691 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.692 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.693 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.694 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.697 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.698 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirroring", 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.701 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.703 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.705 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.707 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "nfs", 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.709 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.710 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.711 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.714 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.715 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.716 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "fail_fs": { 2024-04-20T21:43:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "fail_fs", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-20T21:43:50.719 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.720 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.721 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.722 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.723 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.724 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.725 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.726 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.727 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator": { 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "orchestrator", 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.728 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm", 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "rook", 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "test_orchestrator" 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Orchestrator backend", 2024-04-20T21:43:50.729 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_perf_query", 2024-04-20T21:43:50.730 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.731 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.732 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.734 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.736 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "osd_support", 2024-04-20T21:43:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.741 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.743 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.744 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.745 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.747 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "pg_autoscaler", 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.748 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.749 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.750 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.752 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.753 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.754 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.756 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.758 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "threshold": { 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "threshold", 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.759 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "3.0", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1.0", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "scaling threshold", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.760 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "progress", 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_pg_recovery_event": { 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_pg_recovery_event", 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.762 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:50.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.765 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.772 INFO:teuthology.orchestra.run.smithi007.stdout: "max_completed_events": { 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_completed_events", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "50", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.773 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "number of past completed events to remember", 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "sleep_interval": { 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "sleep_interval", 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.774 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "5", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "how long the module is going to sleep", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "prometheus", 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cache": { 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "cache", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.777 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "exclude_perf_counters": { 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "exclude_perf_counters", 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.779 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.780 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.783 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.784 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.786 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools": { 2024-04-20T21:43:50.787 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.788 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-20T21:43:50.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "300", 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.790 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "scrape_interval": { 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "scrape_interval", 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.791 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15.0", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.792 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.793 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "::", 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "9283", 2024-04-20T21:43:50.795 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.796 INFO:teuthology.orchestra.run.smithi007.stdout: "stale_cache_strategy": { 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stale_cache_strategy", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "log", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.797 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_behaviour": { 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_behaviour", 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.798 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "default", 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default", 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "error" 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.799 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "standby_error_status_code": { 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "standby_error_status_code", 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "500", 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "400", 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "599", 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.804 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.805 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.806 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.807 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.809 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_snap_create": { 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_snap_create", 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.811 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10", 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mirror_snapshot_schedule": { 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mirror_snapshot_schedule", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.813 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "trash_purge_schedule": { 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "trash_purge_schedule", 2024-04-20T21:43:50.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.815 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "restful", 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.816 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_auth": { 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enable_auth", 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.817 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: "key_file": { 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "key_file", 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.822 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.823 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.825 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.826 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.827 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "server_addr": { 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_addr", 2024-04-20T21:43:50.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "server_port": { 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "server_port", 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.830 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rgw", 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.833 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.835 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.837 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.838 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.839 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rook", 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.840 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.841 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.844 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.846 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.848 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "storage_class": { 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "storage_class", 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "local", 2024-04-20T21:43:50.850 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "selftest", 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.852 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.853 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.854 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.856 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.858 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.859 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.860 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "roption1": { 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption1", 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.861 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "roption2": { 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "roption2", 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "xyz", 2024-04-20T21:43:50.863 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.864 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption1": { 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption1", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.865 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption2": { 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption2", 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.866 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.867 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption3": { 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption3", 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "float", 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption4": { 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption4", 2024-04-20T21:43:50.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption5": { 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption5", 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption6": { 2024-04-20T21:43:50.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption6", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.873 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "rwoption7": { 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rwoption7", 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.874 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "1", 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "42", 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "testkey": { 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testkey", 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.876 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "testlkey": { 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testlkey", 2024-04-20T21:43:50.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.878 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "testnewline": { 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "testnewline", 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.879 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_m_granularity": { 2024-04-20T21:43:50.881 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "allow_m_granularity", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "allow minute scheduled snapshots", 2024-04-20T21:43:50.882 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "dump_on_update": { 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "dump_on_update", 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "dump database to debug log on update", 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.885 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.886 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.887 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.888 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.889 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.890 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.892 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "stats", 2024-04-20T21:43:50.893 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.894 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.895 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.897 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.898 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.899 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "status", 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.903 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.904 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.906 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.907 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.908 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telegraf", 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.911 INFO:teuthology.orchestra.run.smithi007.stdout: "address": { 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "address", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.912 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.913 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "15", 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.915 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.916 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.917 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.918 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.919 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.920 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.922 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "telemetry", 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.923 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_basic": { 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_basic", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.924 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_crash": { 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_crash", 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_device": { 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_device", 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_ident": { 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_ident", 2024-04-20T21:43:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "channel_perf": { 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "channel_perf", 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.930 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.931 INFO:teuthology.orchestra.run.smithi007.stdout: "contact": { 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "contact", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.932 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": { 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "description", 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.933 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "device_url": { 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "device_url", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "enabled": { 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "enabled", 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.936 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.937 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "24", 2024-04-20T21:43:50.938 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "8", 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "last_opt_revision": { 2024-04-20T21:43:50.939 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "last_opt_revision", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "1", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.940 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard": { 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard", 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.941 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.942 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "leaderboard_description": { 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "leaderboard_description", 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.943 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.945 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.946 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.947 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.948 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.949 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.950 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.951 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "organization": { 2024-04-20T21:43:50.952 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "organization", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.953 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "proxy": { 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "proxy", 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.954 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.955 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "url": { 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "url", 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-20T21:43:50.956 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "test_orchestrator", 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.958 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.959 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.960 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.961 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.962 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.963 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.964 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.965 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.967 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.968 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.969 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.970 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.971 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.972 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.973 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.974 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "max_concurrent_clones": { 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "max_concurrent_clones", 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "4", 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "periodic_async_work": { 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "periodic_async_work", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.977 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Periodically check for async work", 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_delay": { 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_delay", 2024-04-20T21:43:50.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "0", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "snapshot_clone_no_wait": { 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snapshot_clone_no_wait", 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "True", 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.980 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix", 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "can_run": true, 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "error_string": "", 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "module_options": { 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "discovery_interval": { 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "discovery_interval", 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "uint", 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.982 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "100", 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "identifier": { 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "identifier", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.984 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "interval": { 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "interval", 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "secs", 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "60", 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.986 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "log_level": { 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_level", 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.988 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster": { 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster", 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.989 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.990 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_cluster_level": { 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_cluster_level", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "info", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.991 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [ 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "critical", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "debug", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "error", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "info", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "warning" 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.992 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": { 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "log_to_file", 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "bool", 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "False", 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.994 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_host": { 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_host", 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "", 2024-04-20T21:43:50.995 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.996 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.996 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.996 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.997 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.997 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_port": { 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_port", 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "int", 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "10051", 2024-04-20T21:43:50.998 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "zabbix_sender": { 2024-04-20T21:43:50.999 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "zabbix_sender", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "str", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "level": "advanced", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 0, 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "min": "", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": "", 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "enum_allowed": [], 2024-04-20T21:43:51.000 INFO:teuthology.orchestra.run.smithi007.stdout: "desc": "", 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: "long_desc": "", 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tags": [], 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: "see_also": [] 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:51.001 INFO:teuthology.orchestra.run.smithi007.stdout: "services": { 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "dashboard": "https://172.21.15.7:8443/" 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "always_on_modules": { 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus": [ 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-04-20T21:43:51.002 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific": [ 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-04-20T21:43:51.003 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-04-20T21:43:51.004 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy": [ 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-04-20T21:43:51.005 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "reef": [ 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-04-20T21:43:51.006 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "squid": [ 2024-04-20T21:43:51.007 INFO:teuthology.orchestra.run.smithi007.stdout: "balancer", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "crash", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "devicehealth", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "orchestrator", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscaler", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "progress", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_support", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "status", 2024-04-20T21:43:51.008 INFO:teuthology.orchestra.run.smithi007.stdout: "telemetry", 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: "volumes" 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_failure_osd_epoch": 89, 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: "active_clients": [ 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "devicehealth", 2024-04-20T21:43:51.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1594617530 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "libcephsqlite", 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 585483757 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "rbd_support", 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1765390877 2024-04-20T21:43:51.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1605882153 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "snap_schedule", 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 67645336 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "volumes", 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 324551020 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:51.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-20T21:43:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:51.019 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:51.019 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.019 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.019 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1176261463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:48 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2101138719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:49 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2883663574' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:49 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:49 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:43:51.020 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.021 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.022 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:50 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1176261463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:48 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2101138719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:43:51.023 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:49 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2883663574' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:49 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:49 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.024 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.025 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.026 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:50 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 msgr2=0x7fccfc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 0x7fccfc0a6300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcd1810e310 tx=0x7fcd00005f90 comp rx=0 tx=0).stop 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 msgr2=0x7fcd1819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005e30 tx=0x7fcd08035d90 comp rx=0 tx=0).stop 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 shutdown_connections 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fccfc0a3e40 0x7fccfc0a6300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18103860 0x7fcd1819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd1810c310 0x7fcd1819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 --2- 172.21.15.7:0/2883663574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcd18102f40 0x7fcd1819a750 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:43:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 >> 172.21.15.7:0/2883663574 conn(0x7fcd18077a10 msgr2=0x7fcd18100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:43:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 shutdown_connections 2024-04-20T21:43:51.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:43:48.893+0000 7fcd17e00640 1 -- 172.21.15.7:0/2883663574 wait complete. 2024-04-20T21:43:51.035 DEBUG:tasks.check_counter:Getting stats from a 2024-04-20T21:43:51.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-20T21:43:51.036 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:49 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2883663574' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-20T21:43:51.036 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:49 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.036 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:49 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:43:51.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.037 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.038 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.039 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.039 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.039 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.039 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.039 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:50 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.888 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:43:51.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:51 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:43:51.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:51 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:43:51.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:51 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:51 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:51 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:51 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:43:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:51 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:43:51.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:51 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:51 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:51.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:51 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:52.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:51 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi016 to 177.2M 2024-04-20T21:43:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:51 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi016 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-20T21:43:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:51 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:51 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:52.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:51 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:52.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:52 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:43:52.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:52 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:43:52.878 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:52 smithi007 ceph-mon[16867]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:52.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:52 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:43:52.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:52 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:43:52.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:52 smithi016 ceph-mon[21409]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:52.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:52 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi186 to 176.2M 2024-04-20T21:43:52.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:52 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi186 to 184786739: error parsing value: Value '184786739' is below minimum 939524096 2024-04-20T21:43:52.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:52 smithi186 ceph-mon[22157]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:53.906 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:53 smithi007 ceph-mon[16867]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-20T21:43:53.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:53 smithi016 ceph-mon[21409]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-20T21:43:53.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:53 smithi186 ceph-mon[22157]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-20T21:43:54.280 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:43:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004118310, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:43:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000720831, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 6, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 164, 2024-04-20T21:43:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3344, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 185200, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.030905846, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.021852790, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003767105, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 162, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.040571401, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000250440 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 185200 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 224, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 212, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1728960, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 130464, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-04-20T21:43:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.097044935, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.033455832, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.075701112, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010329686, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 211, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.027620435, 2024-04-20T21:43:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000130902 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1728960, 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-04-20T21:43:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:43:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:43:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:43:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-04-20T21:43:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:43:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 151039, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 32, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-04-20T21:43:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:43:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-04-20T21:43:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 226, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1697384, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-04-20T21:43:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 226, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 226, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1697384, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:54.300 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:55.236 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-20T21:43:55.237 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-20T21:43:55.237 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-20T21:43:55.237 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-20T21:43:55.237 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-20T21:43:55.237 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-20T21:43:55.237 DEBUG:tasks.check_counter:Getting stats from d 2024-04-20T21:43:55.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-20T21:43:55.743 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:43:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:56 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:56 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:56 smithi016 ceph-mon[21409]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-04-20T21:43:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:56 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:56 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:56 smithi186 ceph-mon[22157]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-04-20T21:43:56.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:56 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:56 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:56.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:56 smithi007 ceph-mon[16867]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.471 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:57.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:57.472 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:57 smithi007 ceph-mon[16867]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:57.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:57.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:57.696 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:57 smithi016 ceph-mon[21409]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:57.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-20T21:43:57.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-20T21:43:57.717 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:57 smithi186 ceph-mon[22157]: from='mgr.25208 ' entity='mgr.x' 2024-04-20T21:43:58.120 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:43:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-04-20T21:43:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004604027, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000773180, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:43:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 235, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 221, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1733888, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 135392, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.102044081, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.020973217, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.107268507, 2024-04-20T21:43:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.009754013, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 221, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.029561134, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000133760 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1733888, 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 135392 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 168, 2024-04-20T21:43:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 185808, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.031702550, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.021994820, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.004372050, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:43:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 165, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.044169301, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000267692 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 185808 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:43:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:43:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-04-20T21:43:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:43:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:43:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148714, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-04-20T21:43:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-04-20T21:43:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:43:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 233, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1698294, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 233, 2024-04-20T21:43:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 233, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1698294, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:43:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:43:58.139 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:43:58.523 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:58 smithi007 ceph-mon[16867]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:43:58.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:58 smithi007 ceph-mon[16867]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:43:58.524 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:43:58 smithi007 ceph-mon[16867]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:58.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:58 smithi016 ceph-mon[21409]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:43:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:58 smithi016 ceph-mon[21409]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:43:58.695 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:43:58 smithi016 ceph-mon[21409]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:58 smithi186 ceph-mon[22157]: Adjusting osd_memory_target on smithi007 to 177.5M 2024-04-20T21:43:58.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:58 smithi186 ceph-mon[22157]: Unable to set osd_memory_target on smithi007 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-20T21:43:58.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:43:58 smithi186 ceph-mon[22157]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:43:59.082 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-20T21:43:59.082 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-20T21:43:59.082 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-20T21:43:59.083 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-20T21:43:59.083 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-20T21:43:59.083 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-20T21:43:59.083 DEBUG:tasks.check_counter:Getting stats from g 2024-04-20T21:43:59.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-20T21:43:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:00.532 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:00 smithi007 ceph-mon[16867]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:44:00.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:00 smithi016 ceph-mon[21409]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:44:00.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:00 smithi186 ceph-mon[22157]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 231, 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 217, 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1732800, 2024-04-20T21:44:01.984 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 132832, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.103003083, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.035402708, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.076525095, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010735235, 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:01.985 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 217, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.029092021, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000134064 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.986 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1732800, 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 132832 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 2, 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 164, 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 320, 2024-04-20T21:44:01.987 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 185104, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.035716877, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.024081947, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.003264271, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 161, 2024-04-20T21:44:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.040582886, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000252067 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:01.989 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 185104 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:01.990 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.000599651, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000468854, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:01.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-04-20T21:44:01.992 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:01.993 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:01.995 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:01.996 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 148955, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:01.997 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:01.998 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-04-20T21:44:01.999 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 229, 2024-04-20T21:44:02.000 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1697774, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 229, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 229, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1697774, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:02.001 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:02.002 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:02.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:44:02.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:44:02.002 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:44:02.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:02 smithi016 ceph-mon[21409]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:44:02.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:02 smithi186 ceph-mon[22157]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:44:02.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:02 smithi007 ceph-mon[16867]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-20T21:44:02.894 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-20T21:44:02.894 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-20T21:44:02.895 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-20T21:44:02.895 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-20T21:44:02.895 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-20T21:44:02.895 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-20T21:44:02.895 DEBUG:tasks.check_counter:Getting stats from j 2024-04-20T21:44:02.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-20T21:44:03.384 INFO:teuthology.orchestra.run.smithi007.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.a/config 2024-04-20T21:44:04.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:04 smithi016 ceph-mon[21409]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-20T21:44:04.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:04 smithi186 ceph-mon[22157]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-20T21:44:04.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:04 smithi007 ceph-mon[16867]: pgmap v333: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.004529455, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.000981378, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 234, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 222, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 1731680, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 136864, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.096550998, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.021529713, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.095379511, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.010721385, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 221, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.031222911, 2024-04-20T21:44:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000141280 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 1731680, 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 136864 2024-04-20T21:44:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_messages": 6, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages": 171, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_bytes": 3344, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_bytes": 186432, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_created_connections": 5, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_total_time": 0.033825638, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_send_time": 0.024619507, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_recv_time": 0.004240043, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 169, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.037857384, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000224008 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "msgr_send_encrypted_bytes": 186432 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "cct": { 2024-04-20T21:44:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "total_workers": 0, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "unhealthy_workers": 0 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool": { 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_items": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_bytes": 149745, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_anon_items": 31, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bytes": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_items": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_items": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "pgmap_items": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_co_items": 2, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_1_items": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "unittest_2_items": 0 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "val": 0, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "max": 104857600, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "get_started": 0, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "get": 236, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "get_sum": 1698684, 2024-04-20T21:44:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "get_or_fail_success": 236, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "take": 0, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "take_sum": 0, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "put": 236, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "put_sum": 1698684, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "wait": { 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "avgcount": 0, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "sum": 0.000000000, 2024-04-20T21:44:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "avgtime": 0.000000000 2024-04-20T21:44:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:44:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-20T21:44:05.837 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-20T21:44:06.533 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:06 smithi007 ceph-mon[16867]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:44:06.621 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-20T21:44:06.622 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-20T21:44:06.622 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-20T21:44:06.622 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-20T21:44:06.622 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-20T21:44:06.622 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-20T21:44:06.622 DEBUG:tasks.check_counter:Getting stats from b 2024-04-20T21:44:06.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-20T21:44:06.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:06 smithi016 ceph-mon[21409]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:44:06.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:06 smithi186 ceph-mon[22157]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-20T21:44:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:08 smithi186 ceph-mon[22157]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:08.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:08 smithi186 ceph-mon[22157]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:44:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:08 smithi016 ceph-mon[21409]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:08.746 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:08 smithi016 ceph-mon[21409]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:44:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:08 smithi007 ceph-mon[16867]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-20T21:44:08.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:08 smithi007 ceph-mon[16867]: osdmap e98: 12 total, 12 up, 12 in 2024-04-20T21:44:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4832, 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4347, 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1596854, 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 14135510, 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 16, 2024-04-20T21:44:09.643 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.444604584, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.134322572, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.134637387, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.142095503, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4346, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.440648300, 2024-04-20T21:44:09.644 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000101391 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2637, 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000660588, 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000250 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 367392, 2024-04-20T21:44:09.645 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 208720 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 12991, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 15999, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 4745613, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 17942838, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 150, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:09.646 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 1.106068349, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.393695088, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.704456284, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.137953610, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 15995, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.701223447, 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000106359 2024-04-20T21:44:09.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 18087, 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.003402493, 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000188 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3811984, 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 8063104 2024-04-20T21:44:09.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 3394, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 3528, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2780898, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 5270738, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 8, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 6, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.316134885, 2024-04-20T21:44:09.649 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.099411308, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.131207720, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.084782312, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3527, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.759318510, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000215287 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.650 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2023, 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000410938, 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000203 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1840112, 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 165232 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-04-20T21:44:09.651 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7221, 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 4.597171364, 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000636639 2024-04-20T21:44:09.652 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 841, 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.051102978, 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000060764 2024-04-20T21:44:09.653 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 2043, 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 2039, 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2039, 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 15.913242964, 2024-04-20T21:44:09.654 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.007804434 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 4, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 21, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 1, 2024-04-20T21:44:09.655 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 893, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 26, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 840, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 27, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 856, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 13, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 402, 2024-04-20T21:44:09.656 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 402, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 9, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 2635, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 2519, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:09.657 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 47, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 4, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 9, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 5527, 2024-04-20T21:44:09.658 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 325, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 13424230511, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 6, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 5527, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:09.659 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 1264, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 17, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 1247, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 4263, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 5, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 429, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 3327, 2024-04-20T21:44:09.660 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 6, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 6600, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 5083, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 123, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 913 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-04-20T21:44:09.661 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 10, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:09.662 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 530, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 469, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 61, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-04-20T21:44:09.663 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 17, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 4, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 1, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-04-20T21:44:09.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 8417, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 8417, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-04-20T21:44:09.665 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 13, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 13, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-04-20T21:44:09.666 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 29436601, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 0, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1860, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 99.379108898, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.053429628 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 0 2024-04-20T21:44:09.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 676, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 1167, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 491, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 107, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 125, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 18, 2024-04-20T21:44:09.668 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 893, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 1433, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 540, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 402, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 890, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 488, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 67756, 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 231800 2024-04-20T21:44:09.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 2419, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 2043, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 39, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-04-20T21:44:09.670 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 765, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.892404829, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001166542 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 175, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.043741255, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000249950 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.671 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 61, 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.055024293, 2024-04-20T21:44:09.672 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000902037 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 59, 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017806419, 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000301803 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.673 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1, 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000169943, 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000169943 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-04-20T21:44:09.674 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.675 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 11, 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.002736315, 2024-04-20T21:44:09.676 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000248755 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 105, 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.113826247, 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001084059 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.677 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 11, 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.017471499, 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.092497409 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.678 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 49, 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012206843, 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000249119 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 155, 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 1.048312634, 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.006763307 2024-04-20T21:44:09.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 113, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 11.662657440, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.103209357 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.680 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-04-20T21:44:09.681 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 16, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.164607191, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.010287949 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 5, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.039765610, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.007953122 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.682 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 7, 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.040043131, 2024-04-20T21:44:09.683 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.005720447 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:09.684 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 4, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.001652444, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000413111 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.685 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8, 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.019983527, 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.002497940 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 13, 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.010087308, 2024-04-20T21:44:09.686 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000775946 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 481, 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.770746036, 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001602382 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:09.687 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-04-20T21:44:09.688 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 6, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 4, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 2, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 32, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 16, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 626, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.689 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:09.690 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:09.691 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:09.692 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:09.693 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 350858, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 539, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 1144, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 13, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-04-20T21:44:09.694 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 7936, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 165, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:09.695 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 2638505, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 33725, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:09.696 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 8729, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 25606035, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 37, 2024-04-20T21:44:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 8729, 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8729, 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 219.604119119, 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.025157992 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-04-20T21:44:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 8729, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 9032 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 8729, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 5579, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 3150, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 0, 2024-04-20T21:44:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 11, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 35, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 1878, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 48, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 1, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 987, 2024-04-20T21:44:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 5544, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 22, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 6, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-04-20T21:44:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-04-20T21:44:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 98, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-04-20T21:44:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 13, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 12, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 12, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 351, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 0, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 149 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.705 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 1, 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-04-20T21:44:09.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 10, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 10, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 469, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 469, 2024-04-20T21:44:09.707 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 21213, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 6732476, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:09.708 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 21213, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 21213, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 6732476, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.709 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:09.710 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 8729, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 25653404, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 8729, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 25653404, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.711 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:09.712 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 8729, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 8729, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 8729, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 8729, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.713 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 469, 2024-04-20T21:44:09.714 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 47369, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 469, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 18, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 47369, 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:09.715 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x557071eda460": { 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 112, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:09.716 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 8417, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 25242297, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 8417, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 1860, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 25242185, 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:09.717 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.718 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:09.718 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds.exported=4 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-20T21:44:10.555 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=1 2024-04-20T21:44:10.556 DEBUG:tasks.check_counter:Getting stats from e 2024-04-20T21:44:10.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-20T21:44:10.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:10 smithi186 ceph-mon[22157]: pgmap v337: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:10.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:10 smithi007 ceph-mon[16867]: pgmap v337: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:10.808 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:10.931 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:10 smithi016 ceph-mon[21409]: pgmap v337: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:12.685 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:12 smithi016 ceph-mon[21409]: pgmap v338: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:12.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:12 smithi186 ceph-mon[22157]: pgmap v338: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:12.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:12 smithi007 ceph-mon[16867]: pgmap v338: 97 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 85 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-20T21:44:12.914 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-20T21:44:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 788, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 790, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 238611, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 250648, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.073137510, 2024-04-20T21:44:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.028554940, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.017158313, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.019438416, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 788, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.031888055, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000040467 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 684, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 852, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 168114, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 676929, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 7, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.075892509, 2024-04-20T21:44:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.024253494, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.016964676, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.025467926, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 849, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.059667471, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000070279 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 460048 2024-04-20T21:44:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1020, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 1001, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2032720, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 381936, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-04-20T21:44:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.135735040, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.040030054, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.061770363, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.035502372, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 1001, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.053221754, 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000053168 2024-04-20T21:44:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1777360, 2024-04-20T21:44:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 140256 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3766, 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.109213227, 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000028999 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-04-20T21:44:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-04-20T21:44:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-04-20T21:44:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-04-20T21:44:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-04-20T21:44:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-04-20T21:44:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 13, 2024-04-20T21:44:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-04-20T21:44:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 13, 2024-04-20T21:44:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 13, 2024-04-20T21:44:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-04-20T21:44:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-04-20T21:44:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-04-20T21:44:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 3, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-04-20T21:44:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-04-20T21:44:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-04-20T21:44:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-04-20T21:44:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 31, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-04-20T21:44:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-04-20T21:44:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4226220, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4225646, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 31 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 15, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 15, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 13, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 13, 2024-04-20T21:44:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 13, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 13, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 50736, 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215416 2024-04-20T21:44:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-04-20T21:44:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-04-20T21:44:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-04-20T21:44:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-04-20T21:44:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-04-20T21:44:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-04-20T21:44:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 6, 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 4, 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 5, 2024-04-20T21:44:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 6329, 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 189840, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 63, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8056, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-04-20T21:44:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 66952, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 401, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-04-20T21:44:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-04-20T21:44:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2230, 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2230, 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2230, 2024-04-20T21:44:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 35.692531537, 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.016005619 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2230, 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2231 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2230, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2230, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1440, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-04-20T21:44:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 788, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-04-20T21:44:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-04-20T21:44:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-04-20T21:44:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-04-20T21:44:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 98, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 12, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 5, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 5, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-04-20T21:44:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-04-20T21:44:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-04-20T21:44:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2488, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2199380, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2488, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2488, 2024-04-20T21:44:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2199380, 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2230, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 31921, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2230, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 31921, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2230, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2230, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2230, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2230, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x555a173906e0": { 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-04-20T21:44:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:12.998 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-20T21:44:13.891 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-20T21:44:13.891 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-20T21:44:13.891 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-20T21:44:13.891 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:13.892 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:13.892 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-20T21:44:13.892 DEBUG:tasks.check_counter:Getting stats from h 2024-04-20T21:44:13.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-20T21:44:14.146 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:14.627 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:14 smithi016 ceph-mon[21409]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-20T21:44:14.628 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:14 smithi016 ceph-mon[21409]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:44:14.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:14 smithi186 ceph-mon[22157]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-20T21:44:14.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:14 smithi186 ceph-mon[22157]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:44:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:14 smithi007 ceph-mon[16867]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-20T21:44:14.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:14 smithi007 ceph-mon[16867]: osdmap e99: 12 total, 12 up, 12 in 2024-04-20T21:44:16.421 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-20T21:44:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2291, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2293, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 690396, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 728545, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-04-20T21:44:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.177811717, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.054338752, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.044401934, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.064858170, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2291, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.071625559, 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000031263 2024-04-20T21:44:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 170, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 876, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 458778, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 4, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.010348527, 2024-04-20T21:44:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.006644136, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001688498, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000050713, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 167, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.026723147, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000160018 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 458080 2024-04-20T21:44:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 261, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 242, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1777820, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 140110, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.047689413, 2024-04-20T21:44:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014086520, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.037798653, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005471265, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 242, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.013948832, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000057639 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1777552, 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 139792 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-04-20T21:44:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 1, 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-MDSRank": { 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3832, 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.095712265, 2024-04-20T21:44:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000024977 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "queue_len": 0, 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "complete_latency": { 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "request": 0, 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "reply": 0, 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "reply_latency": { 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.433 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_reply": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "forward": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_commit": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_split": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_merge": 0, 2024-04-20T21:44:16.434 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes": 18, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_top": 0, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_bottom": 18, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_pinned": 18, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_expired": 0, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "inodes_with_caps": 0, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": 0, 2024-04-20T21:44:16.435 INFO:teuthology.orchestra.run.smithi016.stdout: "subtrees": 5, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_hit": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_forward": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_discover": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "traverse_lock": 0, 2024-04-20T21:44:16.436 INFO:teuthology.orchestra.run.smithi016.stdout: "load_cent": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "q": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "exported": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "exported_inodes": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "imported": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "imported_inodes": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_backtrace_fetch": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:16.437 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rfiles": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rbytes": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "root_rsnaps": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_inodes": 0, 2024-04-20T21:44:16.438 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_file_inodes": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_grant": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:16.439 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_caps_dirty": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "process_request_cap_release": 0 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_cache": { 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:16.440 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_try_discover": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_send_discover": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_created": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_enqueued": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:16.441 INFO:teuthology.orchestra.run.smithi016.stdout: "strays_migrated": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_started": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_completed": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:16.442 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_flush": 0, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "ireq_inodestats": 0 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_log": { 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "evlrg": 0, 2024-04-20T21:44:16.443 INFO:teuthology.orchestra.run.smithi016.stdout: "evadd": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "evex": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "evtrm": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "ev": 20, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "evexg": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "evexd": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "segadd": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "segex": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "segtrm": 0, 2024-04-20T21:44:16.444 INFO:teuthology.orchestra.run.smithi016.stdout: "seg": 2, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "segmjr": 2, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "segexg": 0, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "segexd": 0, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "expos": 4194304, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "wrpos": 4237175, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "rdpos": 4236601, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "jlat": { 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.445 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "replayed": 20 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_mem": { 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "ino": 18, 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "ino+": 18, 2024-04-20T21:44:16.446 INFO:teuthology.orchestra.run.smithi016.stdout: "ino-": 0, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dir": 14, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dir+": 14, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dir-": 0, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dn": 18, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dn+": 18, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dn-": 0, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "cap": 0, 2024-04-20T21:44:16.447 INFO:teuthology.orchestra.run.smithi016.stdout: "cap+": 0, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "cap-": 0, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "rss": 51104, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "heap": 215416 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_server": { 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_client_request": 0, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:16.448 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_request": 0, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_client_session": 0, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "handle_peer_request": 0, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "req_create_latency": { 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.449 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getattr_latency": { 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "req_link_latency": { 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookup_latency": { 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.451 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.452 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupino_latency": { 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupname_latency": { 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.454 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "req_lssnap_latency": { 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mkdir_latency": { 2024-04-20T21:44:16.455 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.456 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.456 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.456 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mknod_latency": { 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "req_mksnap_latency": { 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "req_open_latency": { 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "req_readdir_latency": { 2024-04-20T21:44:16.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rename_latency": { 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.459 INFO:teuthology.orchestra.run.smithi016.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmdir_latency": { 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.461 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.462 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setattr_latency": { 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.464 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setlayout_latency": { 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "req_setxattr_latency": { 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.465 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_symlink_latency": { 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "req_unlink_latency": { 2024-04-20T21:44:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.467 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_sessions": { 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: "session_count": 2, 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: "session_add": 4, 2024-04-20T21:44:16.468 INFO:teuthology.orchestra.run.smithi016.stdout: "session_remove": 2, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_open": 3, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "sessions_stale": 0, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "total_load": 0, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "average_load": 0, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "avg_session_uptime": 4059, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.469 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:16.470 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:16.471 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:16.472 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-04-20T21:44:16.473 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 181772, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 60, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:16.474 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 8056, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 168, 2024-04-20T21:44:16.475 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 74988, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 889, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:16.476 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "objecter": { 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_active": 0, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_laggy": 0, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send": 2294, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_send_bytes": 21, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_resend": 3, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_reply": 2294, 2024-04-20T21:44:16.477 INFO:teuthology.orchestra.run.smithi016.stdout: "op_latency": { 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2294, 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 22.875699585, 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.009971970 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "op_inflight": 0, 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "oplen_avg": { 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 2294, 2024-04-20T21:44:16.478 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 2295 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op": 2294, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op_r": 2294, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op_w": 0, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op_rmw": 0, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "op_pg": 0, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_stat": 1492, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_create": 0, 2024-04-20T21:44:16.479 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_read": 800, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_write": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writefull": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_writesame": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_append": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_zero": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_truncate": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_delete": 0, 2024-04-20T21:44:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_mapext": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_getxattr": 1, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_setxattr": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:16.481 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_call": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_watch": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_notify": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls": 0, 2024-04-20T21:44:16.482 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "osdop_other": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_active": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_send": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_resend": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "linger_ping": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_active": 0, 2024-04-20T21:44:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_send": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "poolop_resend": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_active": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_send": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "poolstat_resend": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_active": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_send": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "statfs_resend": 0, 2024-04-20T21:44:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "command_active": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "command_send": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "command_resend": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "map_epoch": 99, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "map_full": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "map_inc": 13, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_sessions": 4, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_open": 4, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_session_close": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_laggy": 0, 2024-04-20T21:44:16.485 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_wr": 0, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_rd": 2, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_del": 0 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "oft": { 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_objs": 0, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_updates": 0, 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: "omap_total_removes": 0 2024-04-20T21:44:16.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "osd-slow-ops": { 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "slow_ops_count": 0 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "purge_queue": { 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing": 0, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:16.487 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_executed": 0, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:16.488 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 2552, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 2223277, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 2552, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2552, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2223277, 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:16.489 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:16.490 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2294, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 42876, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2294, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 42876, 2024-04-20T21:44:16.491 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 1024, 2024-04-20T21:44:16.492 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 2294, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 2294, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 2294, 2024-04-20T21:44:16.493 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 2294, 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:16.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-04-20T21:44:16.495 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-write_buf_throttle-0x561765c726e0": { 2024-04-20T21:44:16.496 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 3758096384, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:16.497 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 0, 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 0, 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:16.498 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-20T21:44:16.694 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:16 smithi016 ceph-mon[21409]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:44:16.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:16 smithi007 ceph-mon[16867]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:44:16.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:16 smithi186 ceph-mon[22157]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-20T21:44:17.131 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-20T21:44:17.132 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-20T21:44:17.132 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-20T21:44:17.132 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:17.132 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:17.132 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-20T21:44:17.132 DEBUG:tasks.check_counter:Getting stats from k 2024-04-20T21:44:17.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-20T21:44:17.381 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.b/config 2024-04-20T21:44:18.623 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:18 smithi016 ceph-mon[21409]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-20T21:44:18.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:18 smithi007 ceph-mon[16867]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-20T21:44:18.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:18 smithi186 ceph-mon[22157]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-20T21:44:19.227 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:19.228 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001133730, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000180513, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:19.230 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 176, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3344, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 265696, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:19.231 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.012932009, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.008008033, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002356727, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 174, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.034935741, 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000200780 2024-04-20T21:44:19.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 265696 2024-04-20T21:44:19.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 238, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 226, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1732768, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 139424, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:19.234 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.048677633, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013371107, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.040202853, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.005719901, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 225, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012627350, 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000056121 2024-04-20T21:44:19.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1732768, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 139424 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.236 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:19.237 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:19.238 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:19.239 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:19.240 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:19.241 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148731, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:19.242 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-04-20T21:44:19.243 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-04-20T21:44:19.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 240, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1699204, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 240, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-04-20T21:44:19.245 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 240, 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1699204, 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-04-20T21:44:19.246 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-04-20T21:44:19.909 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-20T21:44:19.910 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-20T21:44:19.910 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-20T21:44:19.910 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-20T21:44:19.910 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-20T21:44:19.910 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-20T21:44:19.910 DEBUG:tasks.check_counter:Getting stats from c 2024-04-20T21:44:19.910 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-20T21:44:20.190 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:20.576 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:20 smithi186 ceph-mon[22157]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:20.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:20 smithi007 ceph-mon[16867]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:20.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:20 smithi016 ceph-mon[21409]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:22.042 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 113, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 151, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 30572, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 124707, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 16, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:22.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.024426647, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.005392302, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.011944422, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.002369765, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 145, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.032782623, 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000226087 2024-04-20T21:44:22.044 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 83, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000018966, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000228 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 10768, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 16864 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.045 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 66, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 248, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 15296, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 812835, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 38, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.025242810, 2024-04-20T21:44:22.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.009054008, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.013013298, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000510688, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 246, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.038553357, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000156720 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:22.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 69, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000012127, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000175 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 6000, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 469472 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1314, 2024-04-20T21:44:22.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1117, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2070018, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 653861, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 13, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.139160387, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.048629135, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.113928665, 2024-04-20T21:44:22.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.011791251, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1120, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.055865642, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000049880 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1733, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000437003, 2024-04-20T21:44:22.050 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000252 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1789728, 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 161728 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.051 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 151, 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.006464882, 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000042813 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.052 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000013175, 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000006587 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-20T21:44:22.053 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 18, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 6, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 6, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001842577, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000307096 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-20T21:44:22.054 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 12, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 23, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 23, 2024-04-20T21:44:22.055 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 23, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 3, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 3, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 22, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 10, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 8, 2024-04-20T21:44:22.056 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 3, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 1, 2024-04-20T21:44:22.057 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 3, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 28, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 1, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 325, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 13424230511, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 28, 2024-04-20T21:44:22.058 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 14, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 14, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 14, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T21:44:22.059 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 1, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 3, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 1, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 2 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.060 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 4, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:22.061 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-20T21:44:22.062 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-20T21:44:22.063 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 31, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 31, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-20T21:44:22.064 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4226220, 2024-04-20T21:44:22.065 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 29, 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.810264191, 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.027940144 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.066 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 25, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 25, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 23, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 24, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 23, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 23, 2024-04-20T21:44:22.067 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 0, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 3, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 7, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 4, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 52892, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223608 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 20, 2024-04-20T21:44:22.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 18, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 34, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.069 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000385770, 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000192885 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000606062, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000303031 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.073 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.074 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.076 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-20T21:44:22.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.076 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.077 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.078 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000850745, 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000425372 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.079 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:22.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:22.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.084 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.085 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.086 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.087 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-20T21:44:22.088 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 2, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 1, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 632, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:22.089 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:22.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:22.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:22.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:22.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 275673, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 123, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 176, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 2, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:22.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7936, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 165, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 111144, 2024-04-20T21:44:22.095 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 656, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-20T21:44:22.096 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 114, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 45055, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 33, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 114, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 114, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2.036960458, 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.017868074 2024-04-20T21:44:22.097 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 114, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 141 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 114, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 28, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 86, 2024-04-20T21:44:22.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 29, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 23, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-20T21:44:22.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 28, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-04-20T21:44:22.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-20T21:44:22.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-20T21:44:22.102 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-20T21:44:22.103 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 99, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 10, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 10, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 22, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-04-20T21:44:22.104 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 3 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:22.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1489, 2024-04-20T21:44:22.107 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1960573, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1489, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1489, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1960573, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.108 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:22.109 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 114, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 45055, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 114, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 45055, 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:22.110 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-20T21:44:22.111 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 114, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 114, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 114, 2024-04-20T21:44:22.112 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 114, 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:22.113 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:22.114 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:22.115 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55794c9b6460": { 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 31, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 31916, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 31, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:22.116 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 29, 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 31916, 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:22.117 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-20T21:44:22.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:22 smithi007 ceph-mon[16867]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:22.892 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:22 smithi186 ceph-mon[22157]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:22.920 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-20T21:44:22.921 DEBUG:tasks.check_counter:Getting stats from f 2024-04-20T21:44:22.921 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-20T21:44:22.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:22 smithi016 ceph-mon[21409]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-20T21:44:23.169 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:24.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:24 smithi186 ceph-mon[22157]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:24.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:24 smithi007 ceph-mon[16867]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:24.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:24 smithi016 ceph-mon[21409]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:25.042 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1166, 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1168, 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 8751669, 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 370798, 2024-04-20T21:44:25.043 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.131987797, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.037377838, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.039991516, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.042341384, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1166, 2024-04-20T21:44:25.044 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.057620459, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000049417 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:25.045 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 888, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 869, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 10403641, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 340370, 2024-04-20T21:44:25.046 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.111587699, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.033082589, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.061180776, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.024845250, 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:25.047 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 869, 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.041053370, 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000047242 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.048 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1778528, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 141872 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 774, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 944, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 8585385, 2024-04-20T21:44:25.049 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 785755, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.089197610, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.030178188, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.025997001, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.024123318, 2024-04-20T21:44:25.050 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 941, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.056918986, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000060487 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.051 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 540576 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.052 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4309, 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.120896374, 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000028056 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.053 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.054 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.055 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 1329, 2024-04-20T21:44:25.056 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 1329, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 1329, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 6, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2024-04-20T21:44:25.057 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-20T21:44:25.058 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 325, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 13424230511, 2024-04-20T21:44:25.059 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2024-04-20T21:44:25.060 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-20T21:44:25.061 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:25.062 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:25.063 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:25.064 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 8417, 2024-04-20T21:44:25.065 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 13, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-20T21:44:25.066 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 29436601, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 29436489, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.067 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 8417 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 669, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 1143, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 474, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 100, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 118, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 18, 2024-04-20T21:44:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 1329, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 1386, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 57, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 57080, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 215416 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.069 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.070 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.071 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.072 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.073 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.074 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.075 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.076 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.077 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.078 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-20T21:44:25.079 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.080 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.081 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.082 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.083 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.084 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.085 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-20T21:44:25.086 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.087 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-20T21:44:25.088 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 6, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 4, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 5, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 5825, 2024-04-20T21:44:25.089 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:25.090 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:25.091 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:25.092 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-20T21:44:25.093 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 227021, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 835, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:25.094 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7936, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 165, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:25.095 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 2760844, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 31993, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-20T21:44:25.096 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2561, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 26, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 4, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2561, 2024-04-20T21:44:25.097 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2561, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 21.125589500, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.008248961 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2561, 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2562 2024-04-20T21:44:25.098 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2561, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2561, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1558, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2024-04-20T21:44:25.099 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1000, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-20T21:44:25.100 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 2, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-20T21:44:25.101 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-20T21:44:25.102 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-20T21:44:25.103 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 99, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 6, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 6, 2024-04-20T21:44:25.104 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:25.105 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:25.106 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:25.107 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2824, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 27470155, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2824, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2824, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 27470155, 2024-04-20T21:44:25.108 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:25.109 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2561, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 25245523, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2561, 2024-04-20T21:44:25.110 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 25245523, 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:25.111 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2561, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2561, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2561, 2024-04-20T21:44:25.112 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2561, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:25.113 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:25.114 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55c58c4fe6e0": { 2024-04-20T21:44:25.115 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:25.116 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:25.117 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-20T21:44:25.856 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-20T21:44:25.856 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-20T21:44:25.857 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-20T21:44:25.857 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:25.857 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:25.857 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-20T21:44:25.857 DEBUG:tasks.check_counter:Getting stats from i 2024-04-20T21:44:25.857 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-20T21:44:26.099 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:26.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:26 smithi007 ceph-mon[16867]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:26.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:26 smithi016 ceph-mon[21409]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:26.960 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:26 smithi186 ceph-mon[22157]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:27.842 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 91, 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 158, 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 27637, 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 136447, 2024-04-20T21:44:27.843 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 15, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.024753369, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.005736810, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.014383008, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000952533, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 153, 2024-04-20T21:44:27.844 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.039299066, 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000256856 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 130, 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000029594, 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000227 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.845 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 18112, 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 31744 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1487, 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1232, 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2114116, 2024-04-20T21:44:27.846 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 633486, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 10, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.147209083, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.051675178, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.105449851, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.012660087, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1235, 2024-04-20T21:44:27.847 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.074698511, 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000060484 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1979, 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000412651, 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000208 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.848 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1790912, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 163232 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 69, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 240, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 14658, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 838339, 2024-04-20T21:44:27.849 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 39, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 8, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.027019859, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.008939409, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.014253587, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000858236, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 237, 2024-04-20T21:44:27.850 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.044776032, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000188928 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 46, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000006313, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000137 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2496, 2024-04-20T21:44:27.851 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 468848 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:27.852 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 124, 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.006050438, 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000048793 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2024-04-20T21:44:27.853 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000008993, 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000004496 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 23, 2024-04-20T21:44:27.854 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 11, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 11, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.003998623, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000363511 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 12, 2024-04-20T21:44:27.855 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 11, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 28, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 28, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 0, 2024-04-20T21:44:27.856 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 28, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 1, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 8, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 9, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 5, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 33, 2024-04-20T21:44:27.857 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 14, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 8, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 7, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2024-04-20T21:44:27.858 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 3, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 6, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 42, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 325, 2024-04-20T21:44:27.859 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 13424230511, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 6, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 42, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2024-04-20T21:44:27.860 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 42, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 1, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 2, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 7, 2024-04-20T21:44:27.861 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 1 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2024-04-20T21:44:27.862 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 6, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2024-04-20T21:44:27.863 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_path": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_quiesce_inode": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2024-04-20T21:44:27.864 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 20, 2024-04-20T21:44:27.865 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 20, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 2, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2024-04-20T21:44:27.866 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 2, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 2, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 4194304, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 4237175, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2024-04-20T21:44:27.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 13, 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.058858449, 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.004527573 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 28, 2024-04-20T21:44:27.868 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 28, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 0, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 24, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 25, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 28, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 29, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 1, 2024-04-20T21:44:27.869 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 9, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 11, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 2, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 56248, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 223608 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 25, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2024-04-20T21:44:27.870 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 23, 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 26, 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.871 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 5, 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001972234, 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000394446 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.872 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2024-04-20T21:44:27.873 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000712906, 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000356453 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.874 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1, 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000186523, 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000186523 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.875 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.876 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.877 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.878 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.878 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.879 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.880 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.001126960, 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000375653 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.881 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.882 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.883 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.884 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2024-04-20T21:44:27.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.886 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.887 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2024-04-20T21:44:27.888 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.889 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.890 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 2, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 4, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 2, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 2, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 5, 2024-04-20T21:44:27.891 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 2, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 645, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:27.892 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:27.893 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:27.894 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:27.895 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 267448, 2024-04-20T21:44:27.896 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 124, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:27.897 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 7936, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 165, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 121580, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 1177, 2024-04-20T21:44:27.898 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2024-04-20T21:44:27.899 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 102, 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 55374, 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 33, 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 102, 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2024-04-20T21:44:27.900 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 102, 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 4.945904135, 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.048489256 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 102, 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 131 2024-04-20T21:44:27.901 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 102, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 42, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 60, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 0, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 10, 2024-04-20T21:44:27.902 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 0, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 13, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 11, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 6, 2024-04-20T21:44:27.903 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 42, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 20, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2024-04-20T21:44:27.904 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2024-04-20T21:44:27.905 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2024-04-20T21:44:27.906 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 99, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2024-04-20T21:44:27.907 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 13, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 9, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 9, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 27, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 0, 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 2 2024-04-20T21:44:27.908 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2024-04-20T21:44:27.909 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed_ops": 0, 2024-04-20T21:44:27.910 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 1643, 2024-04-20T21:44:27.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1987144, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 1643, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1643, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1987144, 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:27.912 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:27.913 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 102, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 55374, 2024-04-20T21:44:27.914 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 102, 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 55374, 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.915 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:27.916 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 102, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 102, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 102, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 102, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.917 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2024-04-20T21:44:27.918 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.919 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55890296a460": { 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 20, 2024-04-20T21:44:27.920 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 42871, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 20, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 13, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 42871, 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:27.921 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:27.922 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:27.922 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:27.922 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.922 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:27.922 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-20T21:44:28.675 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-20T21:44:28.676 DEBUG:tasks.check_counter:Getting stats from l 2024-04-20T21:44:28.676 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda shell --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-20T21:44:28.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:28 smithi007 ceph-mon[16867]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:28.912 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/mon.c/config 2024-04-20T21:44:28.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:28 smithi016 ceph-mon[21409]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:28.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:28 smithi186 ceph-mon[22157]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:30.663 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:30 smithi186 ceph-mon[22157]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:44:30.697 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 0, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 2, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 0, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 0, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.000965501, 2024-04-20T21:44:30.698 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000000000, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000155603, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.699 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2024-04-20T21:44:30.700 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 242, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 230, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 1733856, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 141984, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.049457405, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.014536196, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.039974301, 2024-04-20T21:44:30.701 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005703777, 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 229, 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012493701, 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000054557 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:30.702 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 1733856, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 141984 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 7, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 180, 2024-04-20T21:44:30.703 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 3504, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 266224, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 3, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.012024569, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.007864180, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.001848383, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2024-04-20T21:44:30.704 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 177, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.038172844, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000215665 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.705 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 266224 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2024-04-20T21:44:30.706 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-20T21:44:30.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2024-04-20T21:44:30.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2024-04-20T21:44:30.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2024-04-20T21:44:30.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 148692, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 30, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2024-04-20T21:44:30.711 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2024-04-20T21:44:30.712 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2024-04-20T21:44:30.713 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 245, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 1699751, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 245, 2024-04-20T21:44:30.714 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 245, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 1699751, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2024-04-20T21:44:30.715 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:30.716 INFO:teuthology.orchestra.run.smithi186.stdout: } 2024-04-20T21:44:30.716 INFO:teuthology.orchestra.run.smithi186.stdout:} 2024-04-20T21:44:30.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:30 smithi007 ceph-mon[16867]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:44:30.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:30 smithi016 ceph-mon[21409]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-20T21:44:31.361 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-20T21:44:31.362 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-20T21:44:31.362 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-20T21:44:31.362 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-20T21:44:31.362 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-20T21:44:31.362 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-20T21:44:31.363 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-20T21:44:31.374 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-20T21:44:31.374 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:44:31.374 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:44:31.450 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-20T21:44:31.450 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:44:31.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:44:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 -- 172.21.15.7:0/4233492389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f435010e610 msgr2=0x7f435010e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 --2- 172.21.15.7:0/4233492389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f435010e610 0x7f435010e9f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4340009930 tx=0x7f434002bc30 comp rx=0 tx=0).stop 2024-04-20T21:44:31.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 -- 172.21.15.7:0/4233492389 shutdown_connections 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 --2- 172.21.15.7:0/4233492389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4350108f10 0x7f43501153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 --2- 172.21.15.7:0/4233492389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108550 0x7f43501089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 --2- 172.21.15.7:0/4233492389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f435010e610 0x7f435010e9f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 -- 172.21.15.7:0/4233492389 >> 172.21.15.7:0/4233492389 conn(0x7f43500764f0 msgr2=0x7f4350078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 -- 172.21.15.7:0/4233492389 shutdown_connections 2024-04-20T21:44:31.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.731+0000 7f4355600640 1 -- 172.21.15.7:0/4233492389 wait complete. 2024-04-20T21:44:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 Processor -- start 2024-04-20T21:44:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 -- start start 2024-04-20T21:44:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108f10 0x7f435019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 0x7f43501a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4350118940 con 0x7f435010e610 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43501187c0 con 0x7f4350108550 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.733+0000 7f4355600640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f4350118ac0 con 0x7f4350108f10 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 0x7f43501a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52506/0 (socket says 172.21.15.7:52506) 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 -- 172.21.15.7:0/1145257499 learned_addr learned my addr 172.21.15.7:0/1145257499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f4347e00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108f10 0x7f435019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:31.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 -- 172.21.15.7:0/1145257499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108f10 msgr2=0x7f435019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108f10 0x7f435019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 -- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 msgr2=0x7f43501a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 0x7f43501a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 -- 172.21.15.7:0/1145257499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501a3db0 con 0x7f4350108550 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434f400640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 0x7f43501a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:31.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.734+0000 7f434ea00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4340009900 tx=0x7f43400042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.735+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340043020 con 0x7f4350108550 2024-04-20T21:44:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.735+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501a4040 con 0x7f4350108550 2024-04-20T21:44:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.735+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43400044d0 con 0x7f4350108550 2024-04-20T21:44:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.735+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340003be0 con 0x7f4350108550 2024-04-20T21:44:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.735+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501ab930 con 0x7f4350108550 2024-04-20T21:44:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.736+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4350069260 con 0x7f4350108550 2024-04-20T21:44:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.738+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f4340046020 con 0x7f4350108550 2024-04-20T21:44:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.740+0000 7f4347400640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 0x7f43340a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:31.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.740+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f43400f26c0 con 0x7f4350108550 2024-04-20T21:44:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.740+0000 7f4347e00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 0x7f43340a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:31.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.741+0000 7f4347e00640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 0x7f43340a6220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4350077440 tx=0x7f4338005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.744+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f434004d020 con 0x7f4350108550 2024-04-20T21:44:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.956+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4350110a00 con 0x7f4350108550 2024-04-20T21:44:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.959+0000 7f4347400640 1 -- 172.21.15.7:0/1145257499 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+22949 (secure 0 0 0) 0x7f43400bef50 con 0x7f4350108550 2024-04-20T21:44:31.958 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:44:31.958 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:44:13.716179+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 msgr2=0x7f43340a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 0x7f43340a6220 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4350077440 tx=0x7f4338005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 msgr2=0x7f435019edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4340009900 tx=0x7f43400042e0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 shutdown_connections 2024-04-20T21:44:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f435010e610 0x7f43501a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4350108f10 0x7f435019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f43340a3d60 0x7f43340a6220 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 --2- 172.21.15.7:0/1145257499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4350108550 0x7f435019edc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 >> 172.21.15.7:0/1145257499 conn(0x7f43500764f0 msgr2=0x7f4350077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 shutdown_connections 2024-04-20T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:31.961+0000 7f4355600640 1 -- 172.21.15.7:0/1145257499 wait complete. 2024-04-20T21:44:31.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:25:34.376573+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:30:50.564513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:30:51.853916+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:44:31.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:44:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 -- 172.21.15.7:0/1159661539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc109290 msgr2=0x7fc8bc110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/1159661539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc109290 0x7fc8bc110e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009be0 tx=0x7fc8b402bef0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 -- 172.21.15.7:0/1159661539 shutdown_connections 2024-04-20T21:44:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/1159661539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc109290 0x7fc8bc110e60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/1159661539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc104b70 0x7fc8bc108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/1159661539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc1041c0 0x7fc8bc1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 -- 172.21.15.7:0/1159661539 >> 172.21.15.7:0/1159661539 conn(0x7fc8bc0ffa00 msgr2=0x7fc8bc101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 -- 172.21.15.7:0/1159661539 shutdown_connections 2024-04-20T21:44:32.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.190+0000 7fc8c0a00640 1 -- 172.21.15.7:0/1159661539 wait complete. 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.191+0000 7fc8c0a00640 1 Processor -- start 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.191+0000 7fc8c0a00640 1 -- start start 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc1041c0 0x7fc8bc1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc109290 0x7fc8bc1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1143a0 con 0x7fc8bc19a640 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114220 con 0x7fc8bc109290 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8c0a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc114520 con 0x7fc8bc1041c0 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bb400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc1041c0 0x7fc8bc1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33836/0 (socket says 172.21.15.7:33836) 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bbe00640 1 -- 172.21.15.7:0/3176564808 learned_addr learned my addr 172.21.15.7:0/3176564808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8baa00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc109290 0x7fc8bc1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bbe00640 1 -- 172.21.15.7:0/3176564808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc1041c0 msgr2=0x7fc8bc1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.192+0000 7fc8bbe00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc1041c0 0x7fc8bc1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8bbe00640 1 -- 172.21.15.7:0/3176564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc109290 msgr2=0x7fc8bc1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8bbe00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc109290 0x7fc8bc1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8bbe00640 1 -- 172.21.15.7:0/3176564808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc19b160 con 0x7fc8bc19a640 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8bbe00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4004af0 tx=0x7fc8b40336b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4043020 con 0x7fc8bc19a640 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8b4003e50 con 0x7fc8bc19a640 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b403e6b0 con 0x7fc8bc19a640 2024-04-20T21:44:32.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc19b3f0 con 0x7fc8bc19a640 2024-04-20T21:44:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.193+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1a75a0 con 0x7fc8bc19a640 2024-04-20T21:44:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.195+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc105690 con 0x7fc8bc19a640 2024-04-20T21:44:32.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.197+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc8b4046020 con 0x7fc8bc19a640 2024-04-20T21:44:32.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.198+0000 7fc8b3e00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 0x7fc8a00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.198+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fc8b40f2910 con 0x7fc8bc19a640 2024-04-20T21:44:32.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.198+0000 7fc8bb400640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 0x7fc8a00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.199+0000 7fc8bb400640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 0x7fc8a00a6250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009530 tx=0x7fc8ac009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.202+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc8b404d030 con 0x7fc8bc19a640 2024-04-20T21:44:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.413+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8bc104b70 con 0x7fc8bc19a640 2024-04-20T21:44:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.414+0000 7fc8b3e00640 1 -- 172.21.15.7:0/3176564808 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fc8b40bf080 con 0x7fc8bc19a640 2024-04-20T21:44:32.413 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 msgr2=0x7fc8a00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 0x7fc8a00a6250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009530 tx=0x7fc8ac009260 comp rx=0 tx=0).stop 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 msgr2=0x7fc8bc19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4004af0 tx=0x7fc8b40336b0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 shutdown_connections 2024-04-20T21:44:32.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8a00a3d90 0x7fc8a00a6250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8bc109290 0x7fc8bc1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8bc19a640 0x7fc8bc19aaf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 --2- 172.21.15.7:0/3176564808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8bc1041c0 0x7fc8bc1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 >> 172.21.15.7:0/3176564808 conn(0x7fc8bc0ffa00 msgr2=0x7fc8bc100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 shutdown_connections 2024-04-20T21:44:32.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.417+0000 7fc8c0a00640 1 -- 172.21.15.7:0/3176564808 wait complete. 2024-04-20T21:44:32.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:44:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 -- 172.21.15.7:0/3730066065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241084a0 msgr2=0x7fa124108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 --2- 172.21.15.7:0/3730066065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241084a0 0x7fa124108900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009930 tx=0x7fa10c02bc30 comp rx=0 tx=0).stop 2024-04-20T21:44:32.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 -- 172.21.15.7:0/3730066065 shutdown_connections 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 --2- 172.21.15.7:0/3730066065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124108e40 0x7fa124115210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 --2- 172.21.15.7:0/3730066065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1241084a0 0x7fa124108900 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 --2- 172.21.15.7:0/3730066065 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 -- 172.21.15.7:0/3730066065 >> 172.21.15.7:0/3730066065 conn(0x7fa124077d70 msgr2=0x7fa124106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 -- 172.21.15.7:0/3730066065 shutdown_connections 2024-04-20T21:44:32.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.667+0000 7fa123200640 1 -- 172.21.15.7:0/3730066065 wait complete. 2024-04-20T21:44:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.668+0000 7fa123200640 1 Processor -- start 2024-04-20T21:44:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.668+0000 7fa123200640 1 -- start start 2024-04-20T21:44:32.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124108e40 0x7fa1241a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1241186f0 con 0x7fa124108e40 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa124118570 con 0x7fa1241084a0 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa123200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa124118870 con 0x7fa12410e3e0 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa122800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa11be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124108e40 0x7fa1241a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.7:52538/0 (socket says 172.21.15.7:52538) 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa122800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12419ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:39602/0 (socket says 172.21.15.7:39602) 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 -- 172.21.15.7:0/3835021152 learned_addr learned my addr 172.21.15.7:0/3835021152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 -- 172.21.15.7:0/3835021152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 msgr2=0x7fa12419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 -- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124108e40 msgr2=0x7fa1241a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124108e40 0x7fa1241a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.669+0000 7fa121e00640 1 -- 172.21.15.7:0/3835021152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12419f210 con 0x7fa1241084a0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa121e00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa11400e9f0 tx=0x7fa11400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11400cd80 con 0x7fa1241084a0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa114010910 con 0x7fa1241084a0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12419f500 con 0x7fa1241084a0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114018990 con 0x7fa1241084a0 2024-04-20T21:44:32.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.670+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241ab7a0 con 0x7fa1241084a0 2024-04-20T21:44:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.671+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12404fcf0 con 0x7fa1241084a0 2024-04-20T21:44:32.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.673+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa114010ac0 con 0x7fa1241084a0 2024-04-20T21:44:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.675+0000 7fa11aa00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 0x7fa1080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.675+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fa1140d0ac0 con 0x7fa1241084a0 2024-04-20T21:44:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.675+0000 7fa11be00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 0x7fa1080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:32.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.676+0000 7fa11be00640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 0x7fa1080a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005ab0 tx=0x7fa10c005fd0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:32.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.680+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa11409cad0 con 0x7fa1241084a0 2024-04-20T21:44:32.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:32 smithi007 ceph-mon[16867]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:32 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1145257499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:32.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:32 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3176564808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:32.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.887+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1241102d0 con 0x7fa1241084a0 2024-04-20T21:44:32.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.889+0000 7fa11aa00640 1 -- 172.21.15.7:0/3835021152 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fa1140a0dd0 con 0x7fa1241084a0 2024-04-20T21:44:32.888 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-20T21:44:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 msgr2=0x7fa1080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 0x7fa1080a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005ab0 tx=0x7fa10c005fd0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 msgr2=0x7fa1241a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa11400e9f0 tx=0x7fa11400eeb0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 shutdown_connections 2024-04-20T21:44:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa12410e3e0 0x7fa12419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa124108e40 0x7fa1241a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa1080a3d20 0x7fa1080a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 --2- 172.21.15.7:0/3835021152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1241084a0 0x7fa1241a48e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 >> 172.21.15.7:0/3835021152 conn(0x7fa124077d70 msgr2=0x7fa124078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 shutdown_connections 2024-04-20T21:44:32.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:32.891+0000 7fa123200640 1 -- 172.21.15.7:0/3835021152 wait complete. 2024-04-20T21:44:32.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:44:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:32 smithi016 ceph-mon[21409]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:32 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1145257499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:32.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:32 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3176564808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:32.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:32 smithi186 ceph-mon[22157]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-20T21:44:32.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:32 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1145257499' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:32.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:32 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3176564808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 -- 172.21.15.7:0/3975299181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613410d690 msgr2=0x7f6134115260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 --2- 172.21.15.7:0/3975299181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613410d690 0x7f6134115260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f612c009be0 tx=0x7f612c02bef0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 -- 172.21.15.7:0/3975299181 shutdown_connections 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 --2- 172.21.15.7:0/3975299181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613410d690 0x7f6134115260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 --2- 172.21.15.7:0/3975299181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6134108f80 0x7f613410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 --2- 172.21.15.7:0/3975299181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61341085d0 0x7f61341089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 -- 172.21.15.7:0/3975299181 >> 172.21.15.7:0/3975299181 conn(0x7f6134077ae0 msgr2=0x7f6134079f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 -- 172.21.15.7:0/3975299181 shutdown_connections 2024-04-20T21:44:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.118+0000 7f6139a00640 1 -- 172.21.15.7:0/3975299181 wait complete. 2024-04-20T21:44:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.119+0000 7f6139a00640 1 Processor -- start 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 -- start start 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613410d690 0x7f61341a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613419ea10 0x7f613419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6134118750 con 0x7f61341085d0 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61341185d0 con 0x7f613410d690 2024-04-20T21:44:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6139a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f61341188d0 con 0x7f613419ea10 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6133e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6133e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60956/0 (socket says 172.21.15.7:60956) 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6133e00640 1 -- 172.21.15.7:0/3132610465 learned_addr learned my addr 172.21.15.7:0/3132610465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.120+0000 7f6133400640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613410d690 0x7f61341a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6139000640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613419ea10 0x7f613419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 -- 172.21.15.7:0/3132610465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613419ea10 msgr2=0x7f613419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613419ea10 0x7f613419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 -- 172.21.15.7:0/3132610465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613410d690 msgr2=0x7f61341a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613410d690 0x7f61341a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 -- 172.21.15.7:0/3132610465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613419f530 con 0x7f61341085d0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6133e00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f612400e9f0 tx=0x7f612400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612400ccc0 con 0x7f61341085d0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612400ce60 con 0x7f61341085d0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613419f820 con 0x7f61341085d0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.121+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124010690 con 0x7f61341085d0 2024-04-20T21:44:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.122+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61341aba20 con 0x7f61341085d0 2024-04-20T21:44:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.123+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613404fcf0 con 0x7f61341085d0 2024-04-20T21:44:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.125+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f61240046c0 con 0x7f61341085d0 2024-04-20T21:44:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.126+0000 7f6130c00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 0x7f61180a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.126+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f61240d0c10 con 0x7f61341085d0 2024-04-20T21:44:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.127+0000 7f6133400640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 0x7f61180a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.127+0000 7f6133400640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 0x7f61180a61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f61341078c0 tx=0x7f6120009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:33.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.131+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f612409d270 con 0x7f61341085d0 2024-04-20T21:44:33.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.327+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6134108f80 con 0x7f61341085d0 2024-04-20T21:44:33.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.328+0000 7f6130c00640 1 -- 172.21.15.7:0/3132610465 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f612409cb70 con 0x7f61341085d0 2024-04-20T21:44:33.327 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-20T21:44:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 msgr2=0x7f61180a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 0x7f61180a61d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f61341078c0 tx=0x7f6120009260 comp rx=0 tx=0).stop 2024-04-20T21:44:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 msgr2=0x7f61341a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f612400e9f0 tx=0x7f612400edf0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 shutdown_connections 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f61180a3d10 0x7f61180a61d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f613410d690 0x7f61341a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f613419ea10 0x7f613419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 --2- 172.21.15.7:0/3132610465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61341085d0 0x7f61341a4860 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 >> 172.21.15.7:0/3132610465 conn(0x7f6134077ae0 msgr2=0x7f6134079e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.330+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 shutdown_connections 2024-04-20T21:44:33.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.331+0000 7f6139a00640 1 -- 172.21.15.7:0/3132610465 wait complete. 2024-04-20T21:44:33.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:44:33.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.568+0000 7fa13d000640 1 -- 172.21.15.7:0/1621126795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 msgr2=0x7fa138104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.568+0000 7fa13d000640 1 --2- 172.21.15.7:0/1621126795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa138104870 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa128009900 tx=0x7fa12802bc30 comp rx=0 tx=0).stop 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 -- 172.21.15.7:0/1621126795 shutdown_connections 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 --2- 172.21.15.7:0/1621126795 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa138111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 --2- 172.21.15.7:0/1621126795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381061d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 --2- 172.21.15.7:0/1621126795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa138104870 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 -- 172.21.15.7:0/1621126795 >> 172.21.15.7:0/1621126795 conn(0x7fa1380ffbf0 msgr2=0x7fa138102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:33.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 -- 172.21.15.7:0/1621126795 shutdown_connections 2024-04-20T21:44:33.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.569+0000 7fa13d000640 1 -- 172.21.15.7:0/1621126795 wait complete. 2024-04-20T21:44:33.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.570+0000 7fa13d000640 1 Processor -- start 2024-04-20T21:44:33.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.570+0000 7fa13d000640 1 -- start start 2024-04-20T21:44:33.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa1381a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa13819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa138114650 con 0x7fa138105d70 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381144d0 con 0x7fa138104490 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa13d000640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa1381147d0 con 0x7fa138106710 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60976/0 (socket says 172.21.15.7:60976) 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa136800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa13819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa12fe00640 1 -- 172.21.15.7:0/1549075844 learned_addr learned my addr 172.21.15.7:0/1549075844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.571+0000 7fa135e00640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa1381a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 -- 172.21.15.7:0/1549075844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 msgr2=0x7fa1381a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa1381a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 -- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 msgr2=0x7fa1381a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 -- 172.21.15.7:0/1549075844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13819b030 con 0x7fa138106710 2024-04-20T21:44:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa136800640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa13819a8b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa130009fd0 tx=0x7fa13000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13000ce20 con 0x7fa138106710 2024-04-20T21:44:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13819b320 con 0x7fa138106710 2024-04-20T21:44:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa130005020 con 0x7fa138106710 2024-04-20T21:44:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130010720 con 0x7fa138106710 2024-04-20T21:44:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.572+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1381a7570 con 0x7fa138106710 2024-04-20T21:44:33.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.573+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa138069260 con 0x7fa138106710 2024-04-20T21:44:33.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.576+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa130005540 con 0x7fa138106710 2024-04-20T21:44:33.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.577+0000 7fa12ea00640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:33.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.577+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fa1300d0e20 con 0x7fa138106710 2024-04-20T21:44:33.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.577+0000 7fa135e00640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:33.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.578+0000 7fa135e00640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa12802efd0 tx=0x7fa128000bc0 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.582+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa13009d480 con 0x7fa138106710 2024-04-20T21:44:33.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3835021152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:33 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3132610465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.802+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1381a7800 con 0x7fa138106710 2024-04-20T21:44:33.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.802+0000 7fa12ea00640 1 -- 172.21.15.7:0/1549075844 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+929 (secure 0 0 0) 0x7fa13009cd80 con 0x7fa138106710 2024-04-20T21:44:33.802 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/3767646163 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1453786296 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2375060083 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/345179439 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2375060083 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1278013541 2024-04-21T21:33:20.988274+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2119469525 2024-04-21T21:22:48.211353+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2119469525 2024-04-21T21:22:48.211353+0000 2024-04-20T21:44:33.803 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1792814241 2024-04-21T21:22:48.211353+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1236425601 2024-04-21T21:22:48.211353+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6801/2055239987 2024-04-21T21:22:14.963843+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1608503169 2024-04-21T21:22:14.963843+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/1751658409 2024-04-21T21:22:14.963843+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2563830614 2024-04-21T21:22:48.211353+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6800/2055239987 2024-04-21T21:22:14.963843+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:0/2403355260 2024-04-21T21:22:14.963843+0000 2024-04-20T21:44:33.804 INFO:teuthology.orchestra.run.smithi007.stderr:listed 16 entries 2024-04-20T21:44:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.805+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 msgr2=0x7fa11c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.805+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa12802efd0 tx=0x7fa128000bc0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.805+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 msgr2=0x7fa13819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.805+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa13819a8b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa130009fd0 tx=0x7fa13000ef10 comp rx=0 tx=0).stop 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 shutdown_connections 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa138106710 0x7fa13819a8b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa138105d70 0x7fa1381a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa11c0a3d20 0x7fa11c0a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 --2- 172.21.15.7:0/1549075844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa138104490 0x7fa1381a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 >> 172.21.15.7:0/1549075844 conn(0x7fa1380ffbf0 msgr2=0x7fa138100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 shutdown_connections 2024-04-20T21:44:33.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:33.806+0000 7fa13d000640 1 -- 172.21.15.7:0/1549075844 wait complete. 2024-04-20T21:44:33.826 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-20T21:44:33.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:44:33.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-20T21:44:33.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3835021152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:33 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3132610465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3835021152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-20T21:44:33.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:33 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3132610465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-20T21:44:34.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:34 smithi186 ceph-mon[22157]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:44:34.716 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:34 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1549075844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:44:34.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:34 smithi007 ceph-mon[16867]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:44:34.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:34 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1549075844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:44:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:34 smithi016 ceph-mon[21409]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-20T21:44:34.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:34 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1549075844' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-20T21:44:35.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:44:35.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:44:35.724 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:44:35.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:44:35.823 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-20T21:44:35.903 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:44:35.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:44:35.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:44:35.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:44:35.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-20T21:44:36.074 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-20T21:44:36.087 INFO:tasks.cephadm:Teardown begin 2024-04-20T21:44:36.088 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:44:36.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:36.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 -- 172.21.15.7:0/4196210605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 msgr2=0x7fc550111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 --2- 172.21.15.7:0/4196210605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc550111060 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc548009be0 tx=0x7fc54802bef0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 -- 172.21.15.7:0/4196210605 shutdown_connections 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 --2- 172.21.15.7:0/4196210605 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc550111060 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 --2- 172.21.15.7:0/4196210605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc550104d70 0x7fc550108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 --2- 172.21.15.7:0/4196210605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 -- 172.21.15.7:0/4196210605 >> 172.21.15.7:0/4196210605 conn(0x7fc5500ffc20 msgr2=0x7fc550102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 -- 172.21.15.7:0/4196210605 shutdown_connections 2024-04-20T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.270+0000 7fc556a00640 1 -- 172.21.15.7:0/4196210605 wait complete. 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.271+0000 7fc556a00640 1 Processor -- start 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 -- start start 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55019a8e0 0x7fc55019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc550114520 con 0x7fc55019a8e0 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501143a0 con 0x7fc5501043c0 2024-04-20T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc556a00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc5501146a0 con 0x7fc550109490 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc54f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc54f400640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49920/0 (socket says 172.21.15.7:49920) 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc54fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc54f400640 1 -- 172.21.15.7:0/1980891195 learned_addr learned my addr 172.21.15.7:0/1980891195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.272+0000 7fc554a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55019a8e0 0x7fc55019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 -- 172.21.15.7:0/1980891195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 msgr2=0x7fc5501a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 -- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55019a8e0 msgr2=0x7fc55019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55019a8e0 0x7fc55019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 -- 172.21.15.7:0/1980891195 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55019b400 con 0x7fc550109490 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54fe00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54f400640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e9a0 tx=0x7fc53c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00cd80 con 0x7fc550109490 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc53c0035f0 con 0x7fc550109490 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55019b6f0 con 0x7fc550109490 2024-04-20T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c010670 con 0x7fc550109490 2024-04-20T21:44:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.273+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5501a78f0 con 0x7fc550109490 2024-04-20T21:44:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.274+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc550069260 con 0x7fc550109490 2024-04-20T21:44:36.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.277+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc53c0037a0 con 0x7fc550109490 2024-04-20T21:44:36.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.279+0000 7fc54cc00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 0x7fc5340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.279+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fc53c0d0a00 con 0x7fc550109490 2024-04-20T21:44:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.279+0000 7fc54fe00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 0x7fc5340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.280+0000 7fc54fe00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 0x7fc5340a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc550103340 tx=0x7fc540005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:36.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.283+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc53c09d060 con 0x7fc550109490 2024-04-20T21:44:36.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.526+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc55019bdc0 con 0x7fc550109490 2024-04-20T21:44:36.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.528+0000 7fc54cc00640 1 -- 172.21.15.7:0/1980891195 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7fc55019bdc0 con 0x7fc550109490 2024-04-20T21:44:36.527 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:44:36.527 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:44:36.528 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 msgr2=0x7fc5340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 0x7fc5340a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc550103340 tx=0x7fc540005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 msgr2=0x7fc5501a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e9a0 tx=0x7fc53c00ee60 comp rx=0 tx=0).stop 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 shutdown_connections 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc55019a8e0 0x7fc55019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc550109490 0x7fc5501a0c20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc5340a3d20 0x7fc5340a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 --2- 172.21.15.7:0/1980891195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5501043c0 0x7fc5501a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 >> 172.21.15.7:0/1980891195 conn(0x7fc5500ffc20 msgr2=0x7fc550101f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:36.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 shutdown_connections 2024-04-20T21:44:36.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.531+0000 7fc556a00640 1 -- 172.21.15.7:0/1980891195 wait complete. 2024-04-20T21:44:36.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:44:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.749+0000 7ffae8c00640 1 -- 172.21.15.7:0/2003268246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4104d70 msgr2=0x7ffae4108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.749+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2003268246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4104d70 0x7ffae4108db0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009860 tx=0x7ffacc02bb70 comp rx=0 tx=0).stop 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 -- 172.21.15.7:0/2003268246 shutdown_connections 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2003268246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4104d70 0x7ffae4108db0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2003268246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae4109490 0x7ffae4111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2003268246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41043c0 0x7ffae41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 -- 172.21.15.7:0/2003268246 >> 172.21.15.7:0/2003268246 conn(0x7ffae40ffc20 msgr2=0x7ffae4102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 -- 172.21.15.7:0/2003268246 shutdown_connections 2024-04-20T21:44:36.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.750+0000 7ffae8c00640 1 -- 172.21.15.7:0/2003268246 wait complete. 2024-04-20T21:44:36.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 Processor -- start 2024-04-20T21:44:36.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 -- start start 2024-04-20T21:44:36.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae41043c0 0x7ffae41a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4109490 0x7ffae41a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4114500 con 0x7ffae419a880 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4114380 con 0x7ffae4109490 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.751+0000 7ffae8c00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7ffae4114680 con 0x7ffae41043c0 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32776/0 (socket says 172.21.15.7:32776) 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 -- 172.21.15.7:0/2470144228 learned_addr learned my addr 172.21.15.7:0/2470144228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:36.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae1e00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae41043c0 0x7ffae41a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffadbe00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4109490 0x7ffae41a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 -- 172.21.15.7:0/2470144228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae41043c0 msgr2=0x7ffae41a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae41043c0 0x7ffae41a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 -- 172.21.15.7:0/2470144228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4109490 msgr2=0x7ffae41a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4109490 0x7ffae41a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 -- 172.21.15.7:0/2470144228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae419b3a0 con 0x7ffae419a880 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffae2800640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00eb10 tx=0x7ffadc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:36.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.752+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc004b20 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.753+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc004cc0 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.753+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae419b690 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.753+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc010760 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.753+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae41a78e0 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.754+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae4069260 con 0x7ffae419a880 2024-04-20T21:44:36.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.756+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7ffadc005020 con 0x7ffae419a880 2024-04-20T21:44:36.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.757+0000 7ffadaa00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 0x7ffac80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:36.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.757+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7ffadc0d0cf0 con 0x7ffae419a880 2024-04-20T21:44:36.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.757+0000 7ffae1e00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 0x7ffac80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:36.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.758+0000 7ffae1e00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 0x7ffac80a6260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffad40096c0 tx=0x7ffad4009260 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:36.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:36.762+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7ffadc09d350 con 0x7ffae419a880 2024-04-20T21:44:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:36 smithi007 ceph-mon[16867]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:36.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:36 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1980891195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:36.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:36 smithi016 ceph-mon[21409]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:36.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:36 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1980891195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:36 smithi186 ceph-mon[22157]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-20T21:44:36.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:36 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1980891195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.010+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffae4104d70 con 0x7ffae419a880 2024-04-20T21:44:37.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.012+0000 7ffadaa00640 1 -- 172.21.15.7:0/2470144228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11869 (secure 0 0 0) 0x7ffadc09cc50 con 0x7ffae419a880 2024-04-20T21:44:37.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:44:37.011 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/766857391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":766857391},{"type":"v1","addr":"172.21.15.7:6839","nonce":766857391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14463,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6841/771888475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6840","nonce":771888475},{"type":"v1","addr":"172.21.15.7:6841","nonce":771888475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1030631519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1030631519},{"type":"v1","addr":"172.21.15.16:6833","nonce":1030631519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3154608825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3154608825},{"type":"v1","addr":"172.21.15.186:6833","nonce":3154608825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3500904086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3500904086},{"type":"v1","addr":"172.21.15.7:6835","nonce":3500904086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2021614032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2021614032},{"type":"v1","addr":"172.21.15.7:6837","nonce":2021614032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-20T21:30:52.886616+0000","modified":"2024-04-20T21:37:26.451816+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24575,"mds_1":24559,"mds_2":24535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6835/2330254135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2330254135},{"type":"v1","addr":"172.21.15.16:6835","nonce":2330254135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.16:6837/4051126294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4051126294},{"type":"v1","addr":"172.21.15.16:6837","nonce":4051126294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24523":{"gid":24523,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/2162825015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":2162825015},{"type":"v1","addr":"172.21.15.186:6835","nonce":2162825015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.186:6837/2315081064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":2315081064},{"type":"v1","addr":"172.21.15.186:6837","nonce":2315081064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24559":{"gid":24559,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.186:6839/2448649165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2448649165},{"type":"v1","addr":"172.21.15.186:6839","nonce":2448649165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6839/518118418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":518118418},{"type":"v1","addr":"172.21.15.16:6839","nonce":518118418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24575,"qdb_cluster":[24575,24535,24559]},"id":1}]} 2024-04-20T21:44:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-20T21:44:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 msgr2=0x7ffac80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 0x7ffac80a6260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffad40096c0 tx=0x7ffad4009260 comp rx=0 tx=0).stop 2024-04-20T21:44:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 msgr2=0x7ffae419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00eb10 tx=0x7ffadc00c2d0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 shutdown_connections 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7ffac80a3da0 0x7ffac80a6260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4109490 0x7ffae41a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae419a880 0x7ffae419ad30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 --2- 172.21.15.7:0/2470144228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffae41043c0 0x7ffae41a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 >> 172.21.15.7:0/2470144228 conn(0x7ffae40ffc20 msgr2=0x7ffae4100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 shutdown_connections 2024-04-20T21:44:37.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.015+0000 7ffae8c00640 1 -- 172.21.15.7:0/2470144228 wait complete. 2024-04-20T21:44:37.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 -- 172.21.15.7:0/4000047077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108f10 msgr2=0x7fbe3c1153d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 --2- 172.21.15.7:0/4000047077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108f10 0x7fbe3c1153d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009930 tx=0x7fbe3402bc20 comp rx=0 tx=0).stop 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 -- 172.21.15.7:0/4000047077 shutdown_connections 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 --2- 172.21.15.7:0/4000047077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108550 0x7fbe3c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 --2- 172.21.15.7:0/4000047077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108f10 0x7fbe3c1153d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 --2- 172.21.15.7:0/4000047077 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 -- 172.21.15.7:0/4000047077 >> 172.21.15.7:0/4000047077 conn(0x7fbe3c0764f0 msgr2=0x7fbe3c078910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:37.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 -- 172.21.15.7:0/4000047077 shutdown_connections 2024-04-20T21:44:37.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.251+0000 7fbe40e00640 1 -- 172.21.15.7:0/4000047077 wait complete. 2024-04-20T21:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 Processor -- start 2024-04-20T21:44:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 -- start start 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108f10 0x7fbe3c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c1a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c1188f0 con 0x7fbe3c10e610 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c118770 con 0x7fbe3c108f10 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.252+0000 7fbe40e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c118a70 con 0x7fbe3c108550 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe3a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c1a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49958/0 (socket says 172.21.15.7:49958) 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 -- 172.21.15.7:0/691920810 learned_addr learned my addr 172.21.15.7:0/691920810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe31400640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108f10 0x7fbe3c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 -- 172.21.15.7:0/691920810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108f10 msgr2=0x7fbe3c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108f10 0x7fbe3c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 -- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 msgr2=0x7fbe3c1a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c1a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 -- 172.21.15.7:0/691920810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c1a3e20 con 0x7fbe3c108550 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe3a800640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.253+0000 7fbe39e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00ebc0 tx=0x7fbe2c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.254+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c010630 con 0x7fbe3c108550 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.254+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2c010cb0 con 0x7fbe3c108550 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.254+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c018bd0 con 0x7fbe3c108550 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.254+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c1a4110 con 0x7fbe3c108550 2024-04-20T21:44:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.254+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1ab9a0 con 0x7fbe3c108550 2024-04-20T21:44:37.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.255+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c069260 con 0x7fbe3c108550 2024-04-20T21:44:37.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.257+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fbe2c0107d0 con 0x7fbe3c108550 2024-04-20T21:44:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.258+0000 7fbe33400640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 0x7fbe200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.258+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fbe2c0d0d10 con 0x7fbe3c108550 2024-04-20T21:44:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.259+0000 7fbe31400640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 0x7fbe200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.259+0000 7fbe31400640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 0x7fbe200a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0775c0 tx=0x7fbe24005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.265+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fbe2c09cd20 con 0x7fbe3c108550 2024-04-20T21:44:37.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.473+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe3c1a0180 con 0x7fbe3c108550 2024-04-20T21:44:37.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.475+0000 7fbe33400640 1 -- 172.21.15.7:0/691920810 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+22949 (secure 0 0 0) 0x7fbe2c0a1020 con 0x7fbe3c108550 2024-04-20T21:44:37.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-20T21:44:37.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"fsid":"8e9da876-ff5b-11ee-bc93-c7b262605968","created":"2024-04-20T21:21:41.203021+0000","modified":"2024-04-20T21:44:13.716179+0000","last_up_change":"2024-04-20T21:29:53.990407+0000","last_in_change":"2024-04-20T21:29:33.924015+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:25:34.376573+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:30:50.564513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:30:51.853916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f182508-4c5f-4296-9117-738a5d55d302","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6803","nonce":2111694387}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6805","nonce":2111694387}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6809","nonce":2111694387}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2111694387},{"type":"v1","addr":"172.21.15.7:6807","nonce":2111694387}]},"public_addr":"172.21.15.7:6803/2111694387","cluster_addr":"172.21.15.7:6805/2111694387","heartbeat_back_addr":"172.21.15.7:6809/2111694387","heartbeat_front_addr":"172.21.15.7:6807/2111694387","state":["exists","up"]},{"osd":1,"uuid":"dbdf4c8d-79a8-4077-9214-86fc2a0d993d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6801","nonce":3476654955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6803","nonce":3476654955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6807","nonce":3476654955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3476654955},{"type":"v1","addr":"172.21.15.16:6805","nonce":3476654955}]},"public_addr":"172.21.15.16:6801/3476654955","cluster_addr":"172.21.15.16:6803/3476654955","heartbeat_back_addr":"172.21.15.16:6807/3476654955","heartbeat_front_addr":"172.21.15.16:6805/3476654955","state":["exists","up"]},{"osd":2,"uuid":"40aadd77-3cf6-4a09-b30a-b038345607eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6801","nonce":1038897606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6803","nonce":1038897606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6807","nonce":1038897606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":1038897606},{"type":"v1","addr":"172.21.15.186:6805","nonce":1038897606}]},"public_addr":"172.21.15.186:6801/1038897606","cluster_addr":"172.21.15.186:6803/1038897606","heartbeat_back_addr":"172.21.15.186:6807/1038897606","heartbeat_front_addr":"172.21.15.186:6805/1038897606","state":["exists","up"]},{"osd":3,"uuid":"3058d99a-73d2-459c-ac66-127d373f59fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6811","nonce":55132163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6813","nonce":55132163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6817","nonce":55132163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":55132163},{"type":"v1","addr":"172.21.15.7:6815","nonce":55132163}]},"public_addr":"172.21.15.7:6811/55132163","cluster_addr":"172.21.15.7:6813/55132163","heartbeat_back_addr":"172.21.15.7:6817/55132163","heartbeat_front_addr":"172.21.15.7:6815/55132163","state":["exists","up"]},{"osd":4,"uuid":"aeed2b10-a459-46cc-b744-38c3c4fb5eb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6809","nonce":215460715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6811","nonce":215460715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6815","nonce":215460715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":215460715},{"type":"v1","addr":"172.21.15.16:6813","nonce":215460715}]},"public_addr":"172.21.15.16:6809/215460715","cluster_addr":"172.21.15.16:6811/215460715","heartbeat_back_addr":"172.21.15.16:6815/215460715","heartbeat_front_addr":"172.21.15.16:6813/215460715","state":["exists","up"]},{"osd":5,"uuid":"19d66739-ba9a-49b9-a99c-7792746f048d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6809","nonce":955879969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6811","nonce":955879969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6815","nonce":955879969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":955879969},{"type":"v1","addr":"172.21.15.186:6813","nonce":955879969}]},"public_addr":"172.21.15.186:6809/955879969","cluster_addr":"172.21.15.186:6811/955879969","heartbeat_back_addr":"172.21.15.186:6815/955879969","heartbeat_front_addr":"172.21.15.186:6813/955879969","state":["exists","up"]},{"osd":6,"uuid":"3b410e07-62a3-487c-b3fb-0e71d0ed25a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6819","nonce":2723605602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6821","nonce":2723605602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6825","nonce":2723605602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2723605602},{"type":"v1","addr":"172.21.15.7:6823","nonce":2723605602}]},"public_addr":"172.21.15.7:6819/2723605602","cluster_addr":"172.21.15.7:6821/2723605602","heartbeat_back_addr":"172.21.15.7:6825/2723605602","heartbeat_front_addr":"172.21.15.7:6823/2723605602","state":["exists","up"]},{"osd":7,"uuid":"af75979c-c869-4b74-a32f-975d039d77ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6817","nonce":4148897395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6819","nonce":4148897395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6823","nonce":4148897395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":4148897395},{"type":"v1","addr":"172.21.15.16:6821","nonce":4148897395}]},"public_addr":"172.21.15.16:6817/4148897395","cluster_addr":"172.21.15.16:6819/4148897395","heartbeat_back_addr":"172.21.15.16:6823/4148897395","heartbeat_front_addr":"172.21.15.16:6821/4148897395","state":["exists","up"]},{"osd":8,"uuid":"d25aec8d-214f-44ed-8166-4b0dfde293d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6817","nonce":3369879966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6819","nonce":3369879966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6823","nonce":3369879966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":3369879966},{"type":"v1","addr":"172.21.15.186:6821","nonce":3369879966}]},"public_addr":"172.21.15.186:6817/3369879966","cluster_addr":"172.21.15.186:6819/3369879966","heartbeat_back_addr":"172.21.15.186:6823/3369879966","heartbeat_front_addr":"172.21.15.186:6821/3369879966","state":["exists","up"]},{"osd":9,"uuid":"f5c3be56-1835-4a5b-8e4b-6c472ccfa4b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6827","nonce":3395112642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6829","nonce":3395112642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6833","nonce":3395112642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":3395112642},{"type":"v1","addr":"172.21.15.7:6831","nonce":3395112642}]},"public_addr":"172.21.15.7:6827/3395112642","cluster_addr":"172.21.15.7:6829/3395112642","heartbeat_back_addr":"172.21.15.7:6833/3395112642","heartbeat_front_addr":"172.21.15.7:6831/3395112642","state":["exists","up"]},{"osd":10,"uuid":"98497028-afaa-4bae-a644-e31beddd7acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6825","nonce":2592476634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6827","nonce":2592476634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6831","nonce":2592476634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2592476634},{"type":"v1","addr":"172.21.15.16:6829","nonce":2592476634}]},"public_addr":"172.21.15.16:6825/2592476634","cluster_addr":"172.21.15.16:6827/2592476634","heartbeat_back_addr":"172.21.15.16:6831/2592476634","heartbeat_front_addr":"172.21.15.16:6829/2592476634","state":["exists","up"]},{"osd":11,"uuid":"2d80cce8-3e29-4afe-b398-5388ebfe1b5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6825","nonce":2885587987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6827","nonce":2885587987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6831","nonce":2885587987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2885587987},{"type":"v1","addr":"172.21.15.186:6829","nonce":2885587987}]},"public_addr":"172.21.15.186:6825/2885587987","cluster_addr":"172.21.15.186:6827/2885587987","heartbeat_back_addr":"172.21.15.186:6831/2885587987","heartbeat_front_addr":"172.21.15.186:6829/2885587987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:24:40.377361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:05.919701+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:25:30.326209+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:03.806772+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:31.118948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:26:56.479032+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:30.646440+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:27:57.768118+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:23.546276+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:28:58.934758+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:26.287245+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:29:52.310337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:0/3767646163":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1453786296":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6800/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/345179439":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2375060083":"2024-04-21T21:33:20.988274+0000","172.21.15.7:0/1278013541":"2024-04-21T21:33:20.988274+0000","172.21.15.7:6801/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2119469525":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1792814241":"2024-04-21T21:22:48.211353+0000","172.21.15.7:0/1236425601":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6801/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1608503169":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/1751658409":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2563830614":"2024-04-21T21:22:48.211353+0000","172.21.15.7:6800/2055239987":"2024-04-21T21:22:14.963843+0000","172.21.15.7:0/2403355260":"2024-04-21T21:22:14.963843+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 msgr2=0x7fbe200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 0x7fbe200a6260 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0775c0 tx=0x7fbe24005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 msgr2=0x7fbe3c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00ebc0 tx=0x7fbe2c00c6a0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 shutdown_connections 2024-04-20T21:44:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fbe200a3da0 0x7fbe200a6260 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c108f10 0x7fbe3c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe3c10e610 0x7fbe3c1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 --2- 172.21.15.7:0/691920810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbe3c108550 0x7fbe3c19ee30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 >> 172.21.15.7:0/691920810 conn(0x7fbe3c0764f0 msgr2=0x7fbe3c077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 shutdown_connections 2024-04-20T21:44:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.479+0000 7fbe40e00640 1 -- 172.21.15.7:0/691920810 wait complete. 2024-04-20T21:44:37.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:44:37.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 -- 172.21.15.7:0/1329960991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8108680 msgr2=0x7fd4e8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1329960991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8108680 0x7fd4e8108ac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009930 tx=0x7fd4d002bc20 comp rx=0 tx=0).stop 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 -- 172.21.15.7:0/1329960991 shutdown_connections 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1329960991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e8109000 0x7fd4e8115520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1329960991 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8108680 0x7fd4e8108ac0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.724+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1329960991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e810a400 0x7fd4e810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.725+0000 7fd4ed800640 1 -- 172.21.15.7:0/1329960991 >> 172.21.15.7:0/1329960991 conn(0x7fd4e8077dd0 msgr2=0x7fd4e8106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.725+0000 7fd4ed800640 1 -- 172.21.15.7:0/1329960991 shutdown_connections 2024-04-20T21:44:37.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.725+0000 7fd4ed800640 1 -- 172.21.15.7:0/1329960991 wait complete. 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 Processor -- start 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 -- start start 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e8108680 0x7fd4e819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810a400 0x7fd4e81a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8118a20 con 0x7fd4e8108680 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e81188a0 con 0x7fd4e810a400 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4ed800640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e8118ba0 con 0x7fd4e8109000 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4de000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.726+0000 7fd4e6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e8108680 0x7fd4e819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4e7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810a400 0x7fd4e81a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.7:49974/0 (socket says 172.21.15.7:49974) 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 -- 172.21.15.7:0/1240706013 learned_addr learned my addr 172.21.15.7:0/1240706013 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 -- 172.21.15.7:0/1240706013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810a400 msgr2=0x7fd4e81a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810a400 0x7fd4e81a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 -- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e8108680 msgr2=0x7fd4e819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e8108680 0x7fd4e819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 -- 172.21.15.7:0/1240706013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a3e60 con 0x7fd4e8109000 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4de000640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ab0 tx=0x7fd4d00042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:37.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.727+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0043020 con 0x7fd4e8109000 2024-04-20T21:44:37.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.728+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d00044d0 con 0x7fd4e8109000 2024-04-20T21:44:37.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.728+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0003b90 con 0x7fd4e8109000 2024-04-20T21:44:37.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.728+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a40f0 con 0x7fd4e8109000 2024-04-20T21:44:37.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.728+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81ab9e0 con 0x7fd4e8109000 2024-04-20T21:44:37.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.729+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e8069260 con 0x7fd4e8109000 2024-04-20T21:44:37.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.731+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fd4d0046020 con 0x7fd4e8109000 2024-04-20T21:44:37.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.732+0000 7fd4dfe00640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 0x7fd4cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:37.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.732+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fd4d00f2530 con 0x7fd4e8109000 2024-04-20T21:44:37.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.732+0000 7fd4e6a00640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 0x7fd4cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:37.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.733+0000 7fd4e6a00640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 0x7fd4cc0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009bd0 tx=0x7fd4d8005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:37.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.736+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fd4d004d020 con 0x7fd4e8109000 2024-04-20T21:44:37.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2470144228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:37 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/691920810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2470144228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:37 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/691920810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2470144228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:37 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/691920810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-20T21:44:37.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:37.988+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd4e8110be0 con 0x7fd4e8109000 2024-04-20T21:44:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.001+0000 7fd4dfe00640 1 -- 172.21.15.7:0/1240706013 <== mon.1 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fd4d00beca0 con 0x7fd4e8109000 2024-04-20T21:44:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:44:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 msgr2=0x7fd4cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 0x7fd4cc0a6260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009bd0 tx=0x7fd4d8005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 msgr2=0x7fd4e819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ab0 tx=0x7fd4d00042e0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 shutdown_connections 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fd4cc0a3da0 0x7fd4cc0a6260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e810a400 0x7fd4e81a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4e8109000 0x7fd4e819f400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 --2- 172.21.15.7:0/1240706013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4e8108680 0x7fd4e819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 >> 172.21.15.7:0/1240706013 conn(0x7fd4e8077dd0 msgr2=0x7fd4e8078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.004+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 shutdown_connections 2024-04-20T21:44:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.005+0000 7fd4ed800640 1 -- 172.21.15.7:0/1240706013 wait complete. 2024-04-20T21:44:38.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:44:38.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.225+0000 7f577b200640 1 -- 172.21.15.7:0/2424924956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076490 msgr2=0x7f577c076870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.225+0000 7f577b200640 1 --2- 172.21.15.7:0/2424924956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076490 0x7f577c076870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f576c009930 tx=0x7f576c02bc20 comp rx=0 tx=0).stop 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 -- 172.21.15.7:0/2424924956 shutdown_connections 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 --2- 172.21.15.7:0/2424924956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 0x7f577c114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 --2- 172.21.15.7:0/2424924956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076db0 0x7f577c1126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 --2- 172.21.15.7:0/2424924956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076490 0x7f577c076870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 -- 172.21.15.7:0/2424924956 >> 172.21.15.7:0/2424924956 conn(0x7f577c0ffa30 msgr2=0x7f577c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:38.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 -- 172.21.15.7:0/2424924956 shutdown_connections 2024-04-20T21:44:38.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.226+0000 7f577b200640 1 -- 172.21.15.7:0/2424924956 wait complete. 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 Processor -- start 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 -- start start 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076db0 0x7f577c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 0x7f577c1a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f577c118780 con 0x7f577c076490 2024-04-20T21:44:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f577c118600 con 0x7f577c076db0 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.227+0000 7f577b200640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7f577c118900 con 0x7f577c112be0 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32838/0 (socket says 172.21.15.7:32838) 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 -- 172.21.15.7:0/211918556 learned_addr learned my addr 172.21.15.7:0/211918556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f577a800640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 0x7f577c1a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779400640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076db0 0x7f577c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 -- 172.21.15.7:0/211918556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 msgr2=0x7f577c1a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 0x7f577c1a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 -- 172.21.15.7:0/211918556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076db0 msgr2=0x7f577c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076db0 0x7f577c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.228+0000 7f5779e00640 1 -- 172.21.15.7:0/211918556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c1a3d20 con 0x7f577c076490 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f5779e00640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f576c005ab0 tx=0x7f576c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c043020 con 0x7f577c076490 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c005000 con 0x7f577c076490 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1a3fb0 con 0x7f577c076490 2024-04-20T21:44:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c003ab0 con 0x7f577c076490 2024-04-20T21:44:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.229+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1ab8a0 con 0x7f577c076490 2024-04-20T21:44:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.230+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577c04fcf0 con 0x7f577c076490 2024-04-20T21:44:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.232+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7f576c046020 con 0x7f577c076490 2024-04-20T21:44:38.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.234+0000 7f5772a00640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 0x7f57600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.235+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8698+0+0 (secure 0 0 0) 0x7f576c0f2720 con 0x7f577c076490 2024-04-20T21:44:38.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.235+0000 7f5779400640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 0x7f57600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:38.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.236+0000 7f5779400640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 0x7f57600a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f577c101f00 tx=0x7f5768005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:38.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.240+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7f576c04d020 con 0x7f577c076490 2024-04-20T21:44:38.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:37 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a[16862]: 2024-04-20T21:44:37.991+0000 7fc48ea00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:44:38.444 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.445 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.445 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mds[51076]: -- 172.21.15.16:0/357380036 <== mon.2 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558062801a40 con 0x5580627d4c00 2024-04-20T21:44:38.445 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: starting mds.e at 2024-04-20T21:44:38.445 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.445 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.446 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: starting mds.b at 2024-04-20T21:44:38.446 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.446 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.446 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: starting mds.h at 2024-04-20T21:44:38.465 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.465 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.465 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mds[53099]: -- 172.21.15.186:0/1864140626 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cb83e7a40 con 0x564cb83bac00 2024-04-20T21:44:38.466 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: starting mds.c at 2024-04-20T21:44:38.466 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.466 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.466 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mds[52788]: -- 172.21.15.186:0/1352397234 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560816371a40 con 0x560816344c00 2024-04-20T21:44:38.466 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: starting mds.f at 2024-04-20T21:44:38.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: ignoring --setuser ceph since I am not root 2024-04-20T21:44:38.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: ignoring --setgroup ceph since I am not root 2024-04-20T21:44:38.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mds[53419]: -- 172.21.15.186:0/957234510 <== mon.1 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558486427a40 con 0x5584863fac00 2024-04-20T21:44:38.467 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: starting mds.i at 2024-04-20T21:44:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.493+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f577c10dd70 con 0x7f577c076490 2024-04-20T21:44:38.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:44:38.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/1240706013' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-20T21:44:38.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:38 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:44:38.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:44:38.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/1240706013' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:38.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-20T21:44:38.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:38 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:44:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:38.998+0000 7f5772a00640 1 -- 172.21.15.7:0/211918556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f576c0bedc0 con 0x7f577c076490 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 msgr2=0x7f57600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 0x7f57600a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f577c101f00 tx=0x7f5768005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 msgr2=0x7f577c19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f576c005ab0 tx=0x7f576c005e80 comp rx=0 tx=0).stop 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 shutdown_connections 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7f57600a3d20 0x7f57600a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f577c076db0 0x7f577c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f577c112be0 0x7f577c1a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 --2- 172.21.15.7:0/211918556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f577c076490 0x7f577c19ed80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 >> 172.21.15.7:0/211918556 conn(0x7f577c0ffa30 msgr2=0x7f577c1143e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 shutdown_connections 2024-04-20T21:44:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.002+0000 7f577b200640 1 -- 172.21.15.7:0/211918556 wait complete. 2024-04-20T21:44:39.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-20T21:44:39.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/1240706013' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:39.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-20T21:44:39.021 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-20T21:44:39.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:44:39.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-20T21:44:39.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: osdmap e100: 12 total, 12 up, 12 in 2024-04-20T21:44:39.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-20T21:44:39.022 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:38 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-20T21:44:39.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:44:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 -- 172.21.15.7:0/2695842012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa354068580 msgr2=0x7fa354104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 --2- 172.21.15.7:0/2695842012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa354068580 0x7fa354104ba0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa344009860 tx=0x7fa34402bb70 comp rx=0 tx=0).stop 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 -- 172.21.15.7:0/2695842012 shutdown_connections 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 --2- 172.21.15.7:0/2695842012 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa35410ae40 0x7fa35410d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 --2- 172.21.15.7:0/2695842012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354100b20 0x7fa3541050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 --2- 172.21.15.7:0/2695842012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa354068580 0x7fa354104ba0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.238+0000 7fa353e00640 1 -- 172.21.15.7:0/2695842012 >> 172.21.15.7:0/2695842012 conn(0x7fa354077e10 msgr2=0x7fa3540fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.239+0000 7fa353e00640 1 -- 172.21.15.7:0/2695842012 shutdown_connections 2024-04-20T21:44:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.239+0000 7fa353e00640 1 -- 172.21.15.7:0/2695842012 wait complete. 2024-04-20T21:44:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.240+0000 7fa353e00640 1 Processor -- start 2024-04-20T21:44:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.240+0000 7fa353e00640 1 -- start start 2024-04-20T21:44:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa354100b20 0x7fa3541a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 0x7fa354109b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541120f0 con 0x7fa354068580 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa354111f70 con 0x7fa35410ae40 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fa354112270 con 0x7fa354100b20 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32852/0 (socket says 172.21.15.7:32852) 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 -- 172.21.15.7:0/3859781169 learned_addr learned my addr 172.21.15.7:0/3859781169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa353400640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 0x7fa354109b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352000640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa354100b20 0x7fa3541a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 -- 172.21.15.7:0/3859781169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa354100b20 msgr2=0x7fa3541a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa354100b20 0x7fa3541a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 -- 172.21.15.7:0/3859781169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 msgr2=0x7fa354109b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.241+0000 7fa352a00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 0x7fa354109b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa352a00640 1 -- 172.21.15.7:0/3859781169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35410a3d0 con 0x7fa354068580 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa353400640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 0x7fa354109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa352a00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa344037040 tx=0x7fa3440045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344041020 con 0x7fa354068580 2024-04-20T21:44:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa34402c9f0 con 0x7fa354068580 2024-04-20T21:44:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35410a660 con 0x7fa354068580 2024-04-20T21:44:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344039040 con 0x7fa354068580 2024-04-20T21:44:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.242+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541a7310 con 0x7fa354068580 2024-04-20T21:44:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.244+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354101680 con 0x7fa354068580 2024-04-20T21:44:39.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.246+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fa344044020 con 0x7fa354068580 2024-04-20T21:44:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.248+0000 7fa34b400640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 0x7fa3380a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.249+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8698+0+0 (secure 0 0 0) 0x7fa3440f1fa0 con 0x7fa354068580 2024-04-20T21:44:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.249+0000 7fa352000640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 0x7fa3380a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.249+0000 7fa352000640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 0x7fa3380a61d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa35419be80 tx=0x7fa340005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:39.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.254+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fa34404b020 con 0x7fa354068580 2024-04-20T21:44:39.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:39.462+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa3541a7510 con 0x7fa354068580 2024-04-20T21:44:39.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: Cluster is now healthy 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:44:39.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6834/855046205,v1:172.21.15.186:6835/855046205] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6834/1008943582,v1:172.21.15.16:6835/1008943582] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6836/3848181184,v1:172.21.15.186:6837/3848181184] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.186:6838/1120740731,v1:172.21.15.186:6839/1120740731] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6836/283564373,v1:172.21.15.16:6837/283564373] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: mds.? [v2:172.21.15.16:6838/2726911725,v1:172.21.15.16:6839/2726911725] up:boot 2024-04-20T21:44:39.946 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: fsmap 12 up:standby 2024-04-20T21:44:39.947 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:39 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:44:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:44:39.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: Cluster is now healthy 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:44:39.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6834/855046205,v1:172.21.15.186:6835/855046205] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6834/1008943582,v1:172.21.15.16:6835/1008943582] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6836/3848181184,v1:172.21.15.186:6837/3848181184] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.186:6838/1120740731,v1:172.21.15.186:6839/1120740731] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6836/283564373,v1:172.21.15.16:6837/283564373] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: mds.? [v2:172.21.15.16:6838/2726911725,v1:172.21.15.16:6839/2726911725] up:boot 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: fsmap 12 up:standby 2024-04-20T21:44:39.967 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:39 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.001+0000 7fa34b400640 1 -- 172.21.15.7:0/3859781169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7fa3440be640 con 0x7fa354068580 2024-04-20T21:44:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:44:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 msgr2=0x7fa3380a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 0x7fa3380a61d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa35419be80 tx=0x7fa340005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 msgr2=0x7fa3541a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa344037040 tx=0x7fa3440045b0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 shutdown_connections 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fa3380a3d10 0x7fa3380a61d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35410ae40 0x7fa354109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa354100b20 0x7fa3541a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 --2- 172.21.15.7:0/3859781169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa354068580 0x7fa3541a0570 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.006+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 >> 172.21.15.7:0/3859781169 conn(0x7fa354077e10 msgr2=0x7fa35410c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.007+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 shutdown_connections 2024-04-20T21:44:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.007+0000 7fa353e00640 1 -- 172.21.15.7:0/3859781169 wait complete. 2024-04-20T21:44:40.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-20T21:44:40.030 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: Cluster is now healthy 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/211918556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-20T21:44:40.031 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='mgr.25208 172.21.15.7:0/3598844050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6834/855046205,v1:172.21.15.186:6835/855046205] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6834/1008943582,v1:172.21.15.16:6835/1008943582] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6836/3848181184,v1:172.21.15.186:6837/3848181184] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.186:6838/1120740731,v1:172.21.15.186:6839/1120740731] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6836/283564373,v1:172.21.15.16:6837/283564373] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: mds.? [v2:172.21.15.16:6838/2726911725,v1:172.21.15.16:6839/2726911725] up:boot 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: fsmap 12 up:standby 2024-04-20T21:44:40.032 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:39 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:40.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:44:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 -- 172.21.15.7:0/2903540215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85410c360 msgr2=0x7fc85410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 --2- 172.21.15.7:0/2903540215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85410c360 0x7fc85410e7d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009ca0 tx=0x7fc84c02c180 comp rx=0 tx=0).stop 2024-04-20T21:44:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 -- 172.21.15.7:0/2903540215 shutdown_connections 2024-04-20T21:44:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 --2- 172.21.15.7:0/2903540215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85410c360 0x7fc85410e7d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 --2- 172.21.15.7:0/2903540215 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854103860 0x7fc85410bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 --2- 172.21.15.7:0/2903540215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc854102f40 0x7fc854103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 -- 172.21.15.7:0/2903540215 >> 172.21.15.7:0/2903540215 conn(0x7fc854077a10 msgr2=0x7fc8540fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 -- 172.21.15.7:0/2903540215 shutdown_connections 2024-04-20T21:44:40.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.247+0000 7fc859e00640 1 -- 172.21.15.7:0/2903540215 wait complete. 2024-04-20T21:44:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 Processor -- start 2024-04-20T21:44:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 -- start start 2024-04-20T21:44:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854102f40 0x7fc8541a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:40.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 0x7fc85419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541120f0 con 0x7fc854103860 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc854111f70 con 0x7fc85410c360 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.248+0000 7fc859e00640 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 -- 0x7fc854112270 con 0x7fc854102f40 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc84be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc858a00640 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854102f40 0x7fc8541a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:40.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 0x7fc85419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc84be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:32860/0 (socket says 172.21.15.7:32860) 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc84be00640 1 -- 172.21.15.7:0/2373523115 learned_addr learned my addr 172.21.15.7:0/2373523115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 -- 172.21.15.7:0/2373523115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854102f40 msgr2=0x7fc8541a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854102f40 0x7fc8541a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 -- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 msgr2=0x7fc8541a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc859400640 1 -- 172.21.15.7:0/2373523115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85419af10 con 0x7fc85410c360 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.249+0000 7fc84be00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:44:40.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc859400640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 0x7fc85419a650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bf0 tx=0x7fc84c035d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c039040 con 0x7fc85410c360 2024-04-20T21:44:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85419b1a0 con 0x7fc85410c360 2024-04-20T21:44:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c044070 con 0x7fc85410c360 2024-04-20T21:44:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0328d0 con 0x7fc85410c360 2024-04-20T21:44:40.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.250+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a7330 con 0x7fc85410c360 2024-04-20T21:44:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.251+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc854069b20 con 0x7fc85410c360 2024-04-20T21:44:40.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.253+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 27) ==== 138776+0+0 (secure 0 0 0) 0x7fc84c032a70 con 0x7fc85410c360 2024-04-20T21:44:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.255+0000 7fc852000640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 0x7fc8380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:44:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.256+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8144+0+0 (secure 0 0 0) 0x7fc84c0f3280 con 0x7fc85410c360 2024-04-20T21:44:40.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.256+0000 7fc858a00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 0x7fc8380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:44:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.256+0000 7fc858a00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 0x7fc8380a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc844009bd0 tx=0x7fc844005f90 comp rx=0 tx=0).ready entity=mgr.25208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:44:40.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.257+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188381 (secure 0 0 0) 0x7fc84c0bfb80 con 0x7fc85410c360 2024-04-20T21:44:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:40.464+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc85404fcf0 con 0x7fc85410c360 2024-04-20T21:44:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:40 smithi016 ceph-mon[21409]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-20T21:44:40.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:40 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:40 smithi016 ceph-mon[21409]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:44:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:40 smithi016 ceph-mon[21409]: from='client.? 172.21.15.7:0/2373523115' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:40.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:40 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:40 smithi186 ceph-mon[22157]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-20T21:44:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:40 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:40 smithi186 ceph-mon[22157]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:44:40.965 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:40 smithi186 ceph-mon[22157]: from='client.? 172.21.15.7:0/2373523115' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:40.966 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:40 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.022+0000 7fc852000640 1 -- 172.21.15.7:0/2373523115 <== mon.2 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7fc85404fcf0 con 0x7fc85410c360 2024-04-20T21:44:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-20T21:44:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.028+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 msgr2=0x7fc8380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.028+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 0x7fc8380a6260 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc844009bd0 tx=0x7fc844005f90 comp rx=0 tx=0).stop 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.028+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 msgr2=0x7fc85419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.028+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 0x7fc85419a650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009bf0 tx=0x7fc84c035d80 comp rx=0 tx=0).stop 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 shutdown_connections 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:6800/2899881467,v1:172.21.15.7:6801/2899881467] conn(0x7fc8380a3da0 0x7fc8380a6260 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85410c360 0x7fc85419a650 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854103860 0x7fc8541a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 --2- 172.21.15.7:0/2373523115 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc854102f40 0x7fc8541a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:44:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 >> 172.21.15.7:0/2373523115 conn(0x7fc854077a10 msgr2=0x7fc854100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:44:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 shutdown_connections 2024-04-20T21:44:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-20T21:44:41.029+0000 7fc859e00640 1 -- 172.21.15.7:0/2373523115 wait complete. 2024-04-20T21:44:41.033 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:40 smithi007 ceph-mon[16867]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-20T21:44:41.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:40 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/3859781169' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:41.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:40 smithi007 ceph-mon[16867]: osdmap e101: 12 total, 12 up, 12 in 2024-04-20T21:44:41.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:40 smithi007 ceph-mon[16867]: from='client.? 172.21.15.7:0/2373523115' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:41.034 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:40 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-20T21:44:41.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:44:41.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:44:41.168 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:44:41.200 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-20T21:44:41.200 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:44:41.250 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:44:41.267 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-20T21:44:41.283 INFO:tasks.cephadm:Stopping all daemons... 2024-04-20T21:44:41.284 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-20T21:44:41.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a 2024-04-20T21:44:41.784 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:41 smithi007 systemd[1]: Stopping Ceph mon.a for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:44:42.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 ceph-mon[16867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 ceph-mon[16867]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 ceph-mon[16867]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:44:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a[16862]: 2024-04-20T21:44:42.076+0000 7fc495800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:44:42.283 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a[16862]: 2024-04-20T21:44:42.076+0000 7fc495800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-20T21:44:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:42 smithi016 ceph-mon[21409]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:42 smithi016 ceph-mon[21409]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:42.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:42 smithi016 ceph-mon[21409]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:44:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:42 smithi186 ceph-mon[22157]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-20T21:44:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:42 smithi186 ceph-mon[22157]: osdmap e102: 12 total, 12 up, 12 in 2024-04-20T21:44:42.465 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:42 smithi186 ceph-mon[22157]: pgmap v357: 1 pgs: 1 active+clean; 577 KiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-20T21:44:42.782 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:42 smithi007 podman[82309]: 2024-04-20 21:44:42.29836168 +0000 UTC m=+0.451926458 container died 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-20T21:44:43.282 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:43 smithi007 podman[82309]: 2024-04-20 21:44:43.056597579 +0000 UTC m=+1.210162355 container cleanup 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-20T21:44:43.286 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:43 smithi007 bash[82309]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a 2024-04-20T21:44:43.783 INFO:journalctl@ceph.mon.a.smithi007.stdout:Apr 20 21:44:43 smithi007 podman[82320]: 2024-04-20 21:44:43.316269041 +0000 UTC m=+0.996598535 container remove 42c2fc8e5ad39abd98f76d94516c9d102c98636fb08b41804cb0c01952d5a082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-20T21:44:43.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.a.service' 2024-04-20T21:44:44.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:44:44.017 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-20T21:44:44.017 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-20T21:44:44.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.b 2024-04-20T21:44:44.444 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:44 smithi016 systemd[1]: Stopping Ceph mon.b for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:44:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:44 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b[21405]: 2024-04-20T21:44:44.626+0000 7f4f750a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:44:44.944 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:44 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b[21405]: 2024-04-20T21:44:44.626+0000 7f4f750a7700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-20T21:44:44.945 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:44 smithi016 podman[57243]: 2024-04-20 21:44:44.763989751 +0000 UTC m=+0.310268109 container died 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T21:44:45.796 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:45 smithi016 podman[57243]: 2024-04-20 21:44:45.538009924 +0000 UTC m=+1.084288279 container cleanup 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T21:44:45.796 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:45 smithi016 bash[57243]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b 2024-04-20T21:44:46.194 INFO:journalctl@ceph.mon.b.smithi016.stdout:Apr 20 21:44:45 smithi016 podman[57253]: 2024-04-20 21:44:45.79802856 +0000 UTC m=+1.017741768 container remove 1ec276f2aa16580215f2bd6260aafd6f3a7b7118538def905a25d2ee658d4a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-b, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T21:44:46.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.b.service' 2024-04-20T21:44:46.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:44:46.359 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-20T21:44:46.359 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-20T21:44:46.360 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.c 2024-04-20T21:44:46.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:46 smithi186 systemd[1]: Stopping Ceph mon.c for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:44:47.215 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:46 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c[22153]: 2024-04-20T21:44:46.938+0000 7f4f5a481700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:44:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:46 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c[22153]: 2024-04-20T21:44:46.938+0000 7f4f5a481700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-20T21:44:47.216 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:47 smithi186 podman[59129]: 2024-04-20 21:44:47.162207226 +0000 UTC m=+0.373925316 container died 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-20T21:44:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:47 smithi186 podman[59129]: 2024-04-20 21:44:47.551695913 +0000 UTC m=+0.763413994 container cleanup 49ba9ff3c1814d545dd2b374e7953df459ab851dd7d62188c04ca6e26ade0714 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T21:44:47.715 INFO:journalctl@ceph.mon.c.smithi186.stdout:Apr 20 21:44:47 smithi186 bash[59129]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mon-c 2024-04-20T21:44:49.853 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mon.c.service' 2024-04-20T21:44:49.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:44:49.968 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-20T21:44:49.968 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-20T21:44:49.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x 2024-04-20T21:44:50.532 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:50 smithi007 systemd[1]: Stopping Ceph mgr.x for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:44:54.009 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:53 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:44:53.593+0000 7f1783600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:44:54.010 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:53 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x[17105]: 2024-04-20T21:44:53.593+0000 7f1783600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-20T21:44:54.010 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:53 smithi007 podman[82422]: 2024-04-20 21:44:53.696096889 +0000 UTC m=+3.217613514 container died 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T21:44:54.282 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:54 smithi007 podman[82422]: 2024-04-20 21:44:54.177853716 +0000 UTC m=+3.699370381 container cleanup 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:44:54.283 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:54 smithi007 bash[82422]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x 2024-04-20T21:44:54.782 INFO:journalctl@ceph.mgr.x.smithi007.stdout:Apr 20 21:44:54 smithi007 podman[82432]: 2024-04-20 21:44:54.421137081 +0000 UTC m=+0.695431013 container remove 5f1fd8eb7db408d680d9cf1de0d83076adad4601dd627ec834aa56cdc8ff9f06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a8c84f5997950fd7d008204839253291d52e6eda, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:44:54.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-20T21:44:55.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:44:55.023 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-20T21:44:55.023 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-20T21:44:55.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.y 2024-04-20T21:44:55.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:44:55 smithi016 systemd[1]: Stopping Ceph mgr.y for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:44:57.694 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:44:57 smithi016 podman[57364]: 2024-04-20 21:44:57.283453598 +0000 UTC m=+1.971098917 container died 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-20T21:45:03.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:45:03 smithi016 podman[57364]: 2024-04-20 21:45:03.188233367 +0000 UTC m=+7.875878676 container cleanup 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T21:45:03.444 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:45:03 smithi016 bash[57364]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y 2024-04-20T21:45:06.624 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:45:06 smithi016 podman[57378]: 2024-04-20 21:45:06.366302403 +0000 UTC m=+8.971873034 container remove 40b3af6b6ee64a8f4e20354ec8fbf3c69fc6b484040e8adf211a1d3e39cbfcb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T21:45:06.624 INFO:journalctl@ceph.mgr.y.smithi016.stdout:Apr 20 21:45:06 smithi016 systemd[1]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-20T21:45:06.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-20T21:45:06.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:06.685 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-20T21:45:06.685 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-20T21:45:06.685 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.z 2024-04-20T21:45:06.965 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:06 smithi186 systemd[1]: Stopping Ceph mgr.z for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:08.465 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:08 smithi186 podman[59242]: 2024-04-20 21:45:08.132285686 +0000 UTC m=+1.254113770 container died 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:45:11.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:11 smithi186 podman[59242]: 2024-04-20 21:45:11.387015308 +0000 UTC m=+4.508843391 container cleanup 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-20T21:45:11.715 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:11 smithi186 bash[59242]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z 2024-04-20T21:45:12.238 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:11 smithi186 podman[59257]: 2024-04-20 21:45:11.989133243 +0000 UTC m=+3.733880992 container remove 875a94fb182f74f2a8f08f69e83d02eaf7f5dd1a0b1b64691f1c89ed2513c62e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-20T21:45:12.238 INFO:journalctl@ceph.mgr.z.smithi186.stdout:Apr 20 21:45:11 smithi186 systemd[1]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-20T21:45:12.241 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-20T21:45:12.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:12.278 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-20T21:45:12.278 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-20T21:45:12.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.0 2024-04-20T21:45:12.782 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:12 smithi007 systemd[1]: Stopping Ceph osd.0 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:13.283 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0[29267]: 2024-04-20T21:45:12.907+0000 7f7404a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:13.283 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0[29267]: 2024-04-20T21:45:12.907+0000 7f7404a00700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-20T21:45:13.283 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:12 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0[29267]: 2024-04-20T21:45:12.907+0000 7f7404a00700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:19.082 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:18 smithi007 podman[82557]: 2024-04-20 21:45:18.810880575 +0000 UTC m=+6.058173170 container died 4500558dd979eb966dc8870ee93e9e325abf55258f31867be919981a8086ab92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:19.482 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:19 smithi007 podman[82557]: 2024-04-20 21:45:19.215386 +0000 UTC m=+6.462678589 container cleanup 4500558dd979eb966dc8870ee93e9e325abf55258f31867be919981a8086ab92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:19.483 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:19 smithi007 bash[82557]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0 2024-04-20T21:45:19.782 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:19 smithi007 podman[82576]: 2024-04-20 21:45:19.483269835 +0000 UTC m=+0.652073439 container remove 4500558dd979eb966dc8870ee93e9e325abf55258f31867be919981a8086ab92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:20.639 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:20 smithi007 podman[82661]: 2024-04-20 21:45:20.211364866 +0000 UTC m=+0.045385999 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:20.639 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:20 smithi007 podman[82661]: 2024-04-20 21:45:20.334279017 +0000 UTC m=+0.168300171 container create 2f6cabf6e5b0575f200ae5829e365be15c0d6966c4a61685e46c7a13705efeda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:20.639 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:20 smithi007 podman[82661]: 2024-04-20 21:45:20.639677212 +0000 UTC m=+0.473698373 container init 2f6cabf6e5b0575f200ae5829e365be15c0d6966c4a61685e46c7a13705efeda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:21.032 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:20 smithi007 podman[82661]: 2024-04-20 21:45:20.652308384 +0000 UTC m=+0.486329544 container start 2f6cabf6e5b0575f200ae5829e365be15c0d6966c4a61685e46c7a13705efeda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-20T21:45:21.033 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:20 smithi007 podman[82661]: 2024-04-20 21:45:20.778533177 +0000 UTC m=+0.612554368 container attach 2f6cabf6e5b0575f200ae5829e365be15c0d6966c4a61685e46c7a13705efeda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-20T21:45:21.783 INFO:journalctl@ceph.osd.0.smithi007.stdout:Apr 20 21:45:21 smithi007 podman[82661]: 2024-04-20 21:45:21.513798753 +0000 UTC m=+1.347819920 container died 2f6cabf6e5b0575f200ae5829e365be15c0d6966c4a61685e46c7a13705efeda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-20T21:45:22.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.0.service' 2024-04-20T21:45:22.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:22.399 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-20T21:45:22.399 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-20T21:45:22.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.1 2024-04-20T21:45:22.694 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:22 smithi016 systemd[1]: Stopping Ceph osd.1 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:23.042 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:22 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1[26005]: 2024-04-20T21:45:22.726+0000 7fdfb0312700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:23.042 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:22 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1[26005]: 2024-04-20T21:45:22.726+0000 7fdfb0312700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-20T21:45:23.043 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:22 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1[26005]: 2024-04-20T21:45:22.726+0000 7fdfb0312700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:28.642 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:28 smithi016 podman[57486]: 2024-04-20 21:45:28.380975066 +0000 UTC m=+5.786623075 container died a30fadec1458d8e397dc57536dd9b07ea33c73332c7c2adc69e171f121010ddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-20T21:45:28.901 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:28 smithi016 podman[57486]: 2024-04-20 21:45:28.735051113 +0000 UTC m=+6.140699108 container cleanup a30fadec1458d8e397dc57536dd9b07ea33c73332c7c2adc69e171f121010ddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:28.901 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:28 smithi016 bash[57486]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1 2024-04-20T21:45:29.194 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:28 smithi016 podman[57515]: 2024-04-20 21:45:28.902997884 +0000 UTC m=+0.511106129 container remove a30fadec1458d8e397dc57536dd9b07ea33c73332c7c2adc69e171f121010ddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T21:45:29.597 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.230480604 +0000 UTC m=+0.018740075 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:29.597 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.345774345 +0000 UTC m=+0.134033869 container create 297b7bf1ed594b69bb6b9a04e5a237babe857a6c720536c8178e242ceddd9139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-20T21:45:29.597 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.599090662 +0000 UTC m=+0.387350143 container init 297b7bf1ed594b69bb6b9a04e5a237babe857a6c720536c8178e242ceddd9139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0) 2024-04-20T21:45:29.944 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.602920282 +0000 UTC m=+0.391179760 container start 297b7bf1ed594b69bb6b9a04e5a237babe857a6c720536c8178e242ceddd9139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:29.944 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.69851717 +0000 UTC m=+0.486776664 container attach 297b7bf1ed594b69bb6b9a04e5a237babe857a6c720536c8178e242ceddd9139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:30.257 INFO:journalctl@ceph.osd.1.smithi016.stdout:Apr 20 21:45:29 smithi016 podman[57604]: 2024-04-20 21:45:29.953751707 +0000 UTC m=+0.742011177 container died 297b7bf1ed594b69bb6b9a04e5a237babe857a6c720536c8178e242ceddd9139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-20T21:45:30.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.1.service' 2024-04-20T21:45:30.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:30.666 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-20T21:45:30.667 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-20T21:45:30.667 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.2 2024-04-20T21:45:30.965 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:30 smithi186 systemd[1]: Stopping Ceph osd.2 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:31.465 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2[27296]: 2024-04-20T21:45:30.998+0000 7f961ede5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:31.466 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2[27296]: 2024-04-20T21:45:30.998+0000 7f961ede5700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-20T21:45:31.466 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2[27296]: 2024-04-20T21:45:30.998+0000 7f961ede5700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:36.965 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:36 smithi186 podman[59376]: 2024-04-20 21:45:36.695837059 +0000 UTC m=+5.823236277 container died 9e90caf32b490032cc1de00292473e1771d6e525bc4018527b995268579427c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T21:45:37.442 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:37 smithi186 podman[59376]: 2024-04-20 21:45:37.225592548 +0000 UTC m=+6.352991766 container cleanup 9e90caf32b490032cc1de00292473e1771d6e525bc4018527b995268579427c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:37.442 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:37 smithi186 bash[59376]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2 2024-04-20T21:45:37.715 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:37 smithi186 podman[59394]: 2024-04-20 21:45:37.444305083 +0000 UTC m=+0.751543421 container remove 9e90caf32b490032cc1de00292473e1771d6e525bc4018527b995268579427c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True) 2024-04-20T21:45:38.132 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:37 smithi186 podman[59491]: 2024-04-20 21:45:37.785511122 +0000 UTC m=+0.018257133 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:38.132 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:37 smithi186 podman[59491]: 2024-04-20 21:45:37.920372125 +0000 UTC m=+0.153118129 container create 8f7cde17dbb28639a6dda8f4320dcdf10965dbfd66a035ffc68bd44256131917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:38.466 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:38 smithi186 podman[59491]: 2024-04-20 21:45:38.199347564 +0000 UTC m=+0.432093580 container init 8f7cde17dbb28639a6dda8f4320dcdf10965dbfd66a035ffc68bd44256131917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T21:45:38.466 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:38 smithi186 podman[59491]: 2024-04-20 21:45:38.202850046 +0000 UTC m=+0.435596059 container start 8f7cde17dbb28639a6dda8f4320dcdf10965dbfd66a035ffc68bd44256131917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True) 2024-04-20T21:45:38.466 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:38 smithi186 podman[59491]: 2024-04-20 21:45:38.298812798 +0000 UTC m=+0.531558815 container attach 8f7cde17dbb28639a6dda8f4320dcdf10965dbfd66a035ffc68bd44256131917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T21:45:38.808 INFO:journalctl@ceph.osd.2.smithi186.stdout:Apr 20 21:45:38 smithi186 podman[59491]: 2024-04-20 21:45:38.508500871 +0000 UTC m=+0.741246877 container died 8f7cde17dbb28639a6dda8f4320dcdf10965dbfd66a035ffc68bd44256131917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-20T21:45:39.136 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.2.service' 2024-04-20T21:45:39.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:39.177 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-20T21:45:39.177 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-20T21:45:39.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.3 2024-04-20T21:45:39.533 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:39 smithi007 systemd[1]: Stopping Ceph osd.3 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:40.283 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3[36040]: 2024-04-20T21:45:39.807+0000 7f67cc200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:40.284 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3[36040]: 2024-04-20T21:45:39.807+0000 7f67cc200700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-20T21:45:40.284 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:39 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3[36040]: 2024-04-20T21:45:39.807+0000 7f67cc200700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:46.119 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:45 smithi007 podman[82780]: 2024-04-20 21:45:45.787905468 +0000 UTC m=+6.121872586 container died 18af50c684311ea0f2c3b7731cfd7c31f94187efd716fdd1c784693c7faab127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-20T21:45:46.501 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:46 smithi007 podman[82780]: 2024-04-20 21:45:46.234054196 +0000 UTC m=+6.568021351 container cleanup 18af50c684311ea0f2c3b7731cfd7c31f94187efd716fdd1c784693c7faab127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T21:45:46.502 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:46 smithi007 bash[82780]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3 2024-04-20T21:45:46.783 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:46 smithi007 podman[82800]: 2024-04-20 21:45:46.502048087 +0000 UTC m=+0.704682755 container remove 18af50c684311ea0f2c3b7731cfd7c31f94187efd716fdd1c784693c7faab127 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True) 2024-04-20T21:45:47.724 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:47 smithi007 podman[82887]: 2024-04-20 21:45:47.309185236 +0000 UTC m=+0.038321965 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:47.725 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:47 smithi007 podman[82887]: 2024-04-20 21:45:47.436342592 +0000 UTC m=+0.165479312 container create 5158983ccc5fceec5350d0e60b2031dc4a28c5835ac58d1aa5755614e99d82b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:47.725 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:47 smithi007 podman[82887]: 2024-04-20 21:45:47.723613501 +0000 UTC m=+0.452750257 container init 5158983ccc5fceec5350d0e60b2031dc4a28c5835ac58d1aa5755614e99d82b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:48.033 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:47 smithi007 podman[82887]: 2024-04-20 21:45:47.736716643 +0000 UTC m=+0.465853398 container start 5158983ccc5fceec5350d0e60b2031dc4a28c5835ac58d1aa5755614e99d82b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:48.033 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:47 smithi007 podman[82887]: 2024-04-20 21:45:47.839132049 +0000 UTC m=+0.568268800 container attach 5158983ccc5fceec5350d0e60b2031dc4a28c5835ac58d1aa5755614e99d82b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:45:48.533 INFO:journalctl@ceph.osd.3.smithi007.stdout:Apr 20 21:45:48 smithi007 podman[82887]: 2024-04-20 21:45:48.203941712 +0000 UTC m=+0.933078458 container died 5158983ccc5fceec5350d0e60b2031dc4a28c5835ac58d1aa5755614e99d82b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:49.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.3.service' 2024-04-20T21:45:49.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:49.233 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-20T21:45:49.233 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-20T21:45:49.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.4 2024-04-20T21:45:49.561 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:49 smithi016 systemd[1]: Stopping Ceph osd.4 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:49.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:49 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4[32813]: 2024-04-20T21:45:49.562+0000 7ff3e951e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:49.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:49 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4[32813]: 2024-04-20T21:45:49.562+0000 7ff3e951e700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-20T21:45:49.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:49 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4[32813]: 2024-04-20T21:45:49.562+0000 7ff3e951e700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:45:55.617 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:55 smithi016 podman[57711]: 2024-04-20 21:45:55.314437705 +0000 UTC m=+5.872644882 container died 3b257c22305bb721b0f9eb89b172fd54696c90c10ac87cbd525525b6d6e6dbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:55.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:55 smithi016 podman[57711]: 2024-04-20 21:45:55.737877599 +0000 UTC m=+6.296084766 container cleanup 3b257c22305bb721b0f9eb89b172fd54696c90c10ac87cbd525525b6d6e6dbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-20T21:45:55.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:55 smithi016 bash[57711]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4 2024-04-20T21:45:56.392 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57731]: 2024-04-20 21:45:56.030829695 +0000 UTC m=+0.675217376 container remove 3b257c22305bb721b0f9eb89b172fd54696c90c10ac87cbd525525b6d6e6dbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-20T21:45:56.691 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57833]: 2024-04-20 21:45:56.297696865 +0000 UTC m=+0.018392608 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:45:56.691 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57833]: 2024-04-20 21:45:56.423641395 +0000 UTC m=+0.144337119 container create 91e715b3057c3de1cf870e3047a7f40e960baf42c108a260bff5d293d4802c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-20T21:45:56.944 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57833]: 2024-04-20 21:45:56.693238913 +0000 UTC m=+0.413934636 container init 91e715b3057c3de1cf870e3047a7f40e960baf42c108a260bff5d293d4802c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:45:56.945 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57833]: 2024-04-20 21:45:56.696917598 +0000 UTC m=+0.417613322 container start 91e715b3057c3de1cf870e3047a7f40e960baf42c108a260bff5d293d4802c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-20T21:45:56.945 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:56 smithi016 podman[57833]: 2024-04-20 21:45:56.793049144 +0000 UTC m=+0.513744877 container attach 91e715b3057c3de1cf870e3047a7f40e960baf42c108a260bff5d293d4802c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T21:45:57.369 INFO:journalctl@ceph.osd.4.smithi016.stdout:Apr 20 21:45:57 smithi016 podman[57833]: 2024-04-20 21:45:57.00523725 +0000 UTC m=+0.725932973 container died 91e715b3057c3de1cf870e3047a7f40e960baf42c108a260bff5d293d4802c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T21:45:57.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.4.service' 2024-04-20T21:45:57.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:45:57.787 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-20T21:45:57.788 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-20T21:45:57.788 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.5 2024-04-20T21:45:58.112 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:45:57 smithi186 systemd[1]: Stopping Ceph osd.5 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:45:58.465 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:45:58 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5[34231]: 2024-04-20T21:45:58.113+0000 7f6f6df7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:45:58.465 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:45:58 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5[34231]: 2024-04-20T21:45:58.113+0000 7f6f6df7b700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-20T21:45:58.466 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:45:58 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5[34231]: 2024-04-20T21:45:58.113+0000 7f6f6df7b700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:03.946 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:03 smithi186 podman[59598]: 2024-04-20 21:46:03.684849864 +0000 UTC m=+5.699320574 container died 970768e2f751380f218aa558cde19821d3ca23500cdddd4c540f6905fe85e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:04.215 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59598]: 2024-04-20 21:46:04.038591447 +0000 UTC m=+6.053062154 container cleanup 970768e2f751380f218aa558cde19821d3ca23500cdddd4c540f6905fe85e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-20T21:46:04.215 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 bash[59598]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5 2024-04-20T21:46:04.698 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59615]: 2024-04-20 21:46:04.373875515 +0000 UTC m=+0.684353168 container remove 970768e2f751380f218aa558cde19821d3ca23500cdddd4c540f6905fe85e8ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T21:46:04.965 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59705]: 2024-04-20 21:46:04.603795105 +0000 UTC m=+0.018161410 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:04.966 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59705]: 2024-04-20 21:46:04.725047221 +0000 UTC m=+0.139413499 container create 32d4b5a0a4f42e03285391fb17ee9331f9f253387af16ebf34ac16167c20c761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-20T21:46:05.282 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59705]: 2024-04-20 21:46:04.986077836 +0000 UTC m=+0.400444117 container init 32d4b5a0a4f42e03285391fb17ee9331f9f253387af16ebf34ac16167c20c761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:05.282 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:04 smithi186 podman[59705]: 2024-04-20 21:46:04.98923359 +0000 UTC m=+0.403599869 container start 32d4b5a0a4f42e03285391fb17ee9331f9f253387af16ebf34ac16167c20c761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:05.282 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:05 smithi186 podman[59705]: 2024-04-20 21:46:05.069374782 +0000 UTC m=+0.483741074 container attach 32d4b5a0a4f42e03285391fb17ee9331f9f253387af16ebf34ac16167c20c761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-20T21:46:05.621 INFO:journalctl@ceph.osd.5.smithi186.stdout:Apr 20 21:46:05 smithi186 podman[59705]: 2024-04-20 21:46:05.28411575 +0000 UTC m=+0.698482043 container died 32d4b5a0a4f42e03285391fb17ee9331f9f253387af16ebf34ac16167c20c761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416) 2024-04-20T21:46:05.919 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.5.service' 2024-04-20T21:46:05.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:05.958 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-20T21:46:05.958 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-20T21:46:05.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.6 2024-04-20T21:46:06.533 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:06 smithi007 systemd[1]: Stopping Ceph osd.6 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:07.033 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6[43007]: 2024-04-20T21:46:06.570+0000 7fc542600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:07.033 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6[43007]: 2024-04-20T21:46:06.570+0000 7fc542600700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-20T21:46:07.033 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6[43007]: 2024-04-20T21:46:06.570+0000 7fc542600700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:12.633 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:12 smithi007 podman[83004]: 2024-04-20 21:46:12.322160081 +0000 UTC m=+5.891864111 container died 7adaac2fb1e73c014a4d63e179557e109d32f7f8699ca7c0f55b09925c9ac139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-20T21:46:13.020 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:12 smithi007 podman[83004]: 2024-04-20 21:46:12.769455365 +0000 UTC m=+6.339159413 container cleanup 7adaac2fb1e73c014a4d63e179557e109d32f7f8699ca7c0f55b09925c9ac139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T21:46:13.020 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:12 smithi007 bash[83004]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6 2024-04-20T21:46:13.282 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:13 smithi007 podman[83034]: 2024-04-20 21:46:13.020572279 +0000 UTC m=+0.679842651 container remove 7adaac2fb1e73c014a4d63e179557e109d32f7f8699ca7c0f55b09925c9ac139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-20T21:46:14.229 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:13 smithi007 podman[83126]: 2024-04-20 21:46:13.799852344 +0000 UTC m=+0.044355093 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:14.230 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:13 smithi007 podman[83126]: 2024-04-20 21:46:13.929866861 +0000 UTC m=+0.174369599 container create eb92f329a3f8b32a419e9e922ce0f94492cb7786c346b6b86941bfd19bc64fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:14.230 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:14 smithi007 podman[83126]: 2024-04-20 21:46:14.23025245 +0000 UTC m=+0.474755222 container init eb92f329a3f8b32a419e9e922ce0f94492cb7786c346b6b86941bfd19bc64fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-20T21:46:14.532 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:14 smithi007 podman[83126]: 2024-04-20 21:46:14.241844685 +0000 UTC m=+0.486347477 container start eb92f329a3f8b32a419e9e922ce0f94492cb7786c346b6b86941bfd19bc64fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-04-20T21:46:14.533 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:14 smithi007 podman[83126]: 2024-04-20 21:46:14.316071833 +0000 UTC m=+0.560574571 container attach eb92f329a3f8b32a419e9e922ce0f94492cb7786c346b6b86941bfd19bc64fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:15.016 INFO:journalctl@ceph.osd.6.smithi007.stdout:Apr 20 21:46:14 smithi007 podman[83126]: 2024-04-20 21:46:14.716447446 +0000 UTC m=+0.960950218 container died eb92f329a3f8b32a419e9e922ce0f94492cb7786c346b6b86941bfd19bc64fd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-20T21:46:15.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.6.service' 2024-04-20T21:46:15.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:15.572 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-20T21:46:15.572 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-20T21:46:15.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.7 2024-04-20T21:46:15.897 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:15 smithi016 systemd[1]: Stopping Ceph osd.7 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:16.194 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7[39375]: 2024-04-20T21:46:15.899+0000 7fc8f0d41700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:16.194 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7[39375]: 2024-04-20T21:46:15.899+0000 7fc8f0d41700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-20T21:46:16.194 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:15 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7[39375]: 2024-04-20T21:46:15.899+0000 7fc8f0d41700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:19.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:19 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:19.606+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:19.965 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:19 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:19.702+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:20.283 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:19 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:19.837+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:20.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:20 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:20.574+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:21.215 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:20 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:20.736+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:21.282 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:20 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:20.819+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:21.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:21 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:21.616+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:21.695 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:21 smithi016 podman[57942]: 2024-04-20 21:46:21.436683836 +0000 UTC m=+5.665911407 container died b8bc6d42c0ae585da6a626537736144931243a405c257134eecd6f22ef51c02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-20T21:46:22.010 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:21 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:21.737+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:22.073 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:21 smithi016 podman[57942]: 2024-04-20 21:46:21.831975664 +0000 UTC m=+6.061203238 container cleanup b8bc6d42c0ae585da6a626537736144931243a405c257134eecd6f22ef51c02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:22.073 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:21 smithi016 bash[57942]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7 2024-04-20T21:46:22.282 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:21 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:21.811+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:22.397 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[57961]: 2024-04-20 21:46:22.075155285 +0000 UTC m=+0.634554124 container remove b8bc6d42c0ae585da6a626537736144931243a405c257134eecd6f22ef51c02d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-20T21:46:22.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:22 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:22.011+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:22.677 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:22 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:22.580+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:22.677 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.303004647 +0000 UTC m=+0.018242664 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:22.677 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.409414347 +0000 UTC m=+0.124652395 container create 76bcc11e6182ff017aae0bd97d512f81dcdaf657888b2c2ea3e74909d07795df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T21:46:22.677 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.679005362 +0000 UTC m=+0.394243372 container init 76bcc11e6182ff017aae0bd97d512f81dcdaf657888b2c2ea3e74909d07795df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:46:22.944 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.682648434 +0000 UTC m=+0.397886438 container start 76bcc11e6182ff017aae0bd97d512f81dcdaf657888b2c2ea3e74909d07795df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-20T21:46:22.945 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.770238174 +0000 UTC m=+0.485476187 container attach 76bcc11e6182ff017aae0bd97d512f81dcdaf657888b2c2ea3e74909d07795df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:23.048 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:22 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:22.780+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:23.271 INFO:journalctl@ceph.osd.7.smithi016.stdout:Apr 20 21:46:22 smithi016 podman[58050]: 2024-04-20 21:46:22.984134686 +0000 UTC m=+0.699372690 container died 76bcc11e6182ff017aae0bd97d512f81dcdaf657888b2c2ea3e74909d07795df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-20T21:46:23.282 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:22 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:22.806+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:23.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:23.049+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:23.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.7.service' 2024-04-20T21:46:23.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:23.645 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-20T21:46:23.645 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-20T21:46:23.645 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.8 2024-04-20T21:46:23.873 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:23 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:23.563+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:23.965 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:23 smithi186 systemd[1]: Stopping Ceph osd.8 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:23.965 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:23.825+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:24.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:23 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:23.769+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:24.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:24 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:24.093+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:24.465 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:23.967+0000 7f13583d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:24.465 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:23.967+0000 7f13583d8700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-20T21:46:24.466 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:23 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:23.967+0000 7f13583d8700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:24.842 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:24 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:24.546+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:25.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:24 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:24.744+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:25.108 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:24 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:24.811+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:25.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:25 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:25.110+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:25.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:25 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:25.527+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:26.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:25 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:25.739+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:26.125 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:25 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:25.781+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:26.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:26 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:26.127+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:26.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:26 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:26.547+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:27.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:26 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:26.698+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:27.138 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:26 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:26.777+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:27.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:27 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:27.139+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:27.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:27 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:27.540+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:28.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:27 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:27.744+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:28.153 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:27 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:27.771+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:28.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:28 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:28.154+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:28.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:28 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:28.543+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:28.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:28 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:28.543+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:29.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:28 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:28.700+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:29.201 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:28 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8[40777]: 2024-04-20T21:46:28.733+0000 7f13549cc700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.071368+0000 front 2024-04-20T21:45:17.071459+0000 (oldest deadline 2024-04-20T21:46:19.371277+0000) 2024-04-20T21:46:29.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:29 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:29.202+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:29.908 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:29 smithi186 podman[59824]: 2024-04-20 21:46:29.60523096 +0000 UTC m=+5.763073229 container died 97d706d9c4b9686710e9a3b29dda8cb0d04835ee3b97cababcc377bac8de7930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:46:29.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:29 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:29.578+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:29.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:29 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:29.578+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:30.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:29 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:29.705+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:30.207 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59824]: 2024-04-20 21:46:30.001152937 +0000 UTC m=+6.158995214 container cleanup 97d706d9c4b9686710e9a3b29dda8cb0d04835ee3b97cababcc377bac8de7930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T21:46:30.207 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 bash[59824]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8 2024-04-20T21:46:30.465 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59841]: 2024-04-20 21:46:30.277614653 +0000 UTC m=+0.670296436 container remove 97d706d9c4b9686710e9a3b29dda8cb0d04835ee3b97cababcc377bac8de7930 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5) 2024-04-20T21:46:30.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:30 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:30.208+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:30.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:30 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:30.572+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:30.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:30 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:30.572+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:30.954 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59931]: 2024-04-20 21:46:30.61212414 +0000 UTC m=+0.111176334 container create 884588e4c61d5d1c6b5af73cb6554b16a90365bd8182dc64980f7d3949f764bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-20T21:46:30.954 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59931]: 2024-04-20 21:46:30.519187943 +0000 UTC m=+0.018240168 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:30.955 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59931]: 2024-04-20 21:46:30.85636106 +0000 UTC m=+0.355413267 container init 884588e4c61d5d1c6b5af73cb6554b16a90365bd8182dc64980f7d3949f764bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:30.955 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59931]: 2024-04-20 21:46:30.859348462 +0000 UTC m=+0.358400662 container start 884588e4c61d5d1c6b5af73cb6554b16a90365bd8182dc64980f7d3949f764bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:30.955 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:30 smithi186 podman[59931]: 2024-04-20 21:46:30.956191011 +0000 UTC m=+0.455243214 container attach 884588e4c61d5d1c6b5af73cb6554b16a90365bd8182dc64980f7d3949f764bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:31.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:30 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:30.688+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:31.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:30 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:30.688+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:31.216 INFO:journalctl@ceph.osd.8.smithi186.stdout:Apr 20 21:46:31 smithi186 podman[59931]: 2024-04-20 21:46:31.150995254 +0000 UTC m=+0.650047460 container died 884588e4c61d5d1c6b5af73cb6554b16a90365bd8182dc64980f7d3949f764bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-20T21:46:31.482 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:31.230+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:31.483 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:31 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:31.230+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:31.793 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.8.service' 2024-04-20T21:46:31.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:31.831 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-20T21:46:31.831 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-20T21:46:31.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.9 2024-04-20T21:46:31.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:31 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:31.524+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:31.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:31 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:31.524+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:32.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:31 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:31.726+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:32.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:31 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:31.726+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:32.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:31 smithi007 systemd[1]: Stopping Ceph osd.9 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:32.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:32 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:32.200+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:32.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:32 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:32.200+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:32.732 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:32 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:32.453+0000 7f46ef400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:32.733 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:32 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:32.453+0000 7f46ef400700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-20T21:46:32.733 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:32 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:32.453+0000 7f46ef400700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:32.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:32 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:32.534+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:32.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:32 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:32.534+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:33.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:32 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:32.732+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:33.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:32 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:32.732+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:33.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:33 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:33.203+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:33.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:33 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:33.203+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:33.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:33 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:33.506+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:33.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:33 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:33.506+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:34.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:33 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:33.719+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:34.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:33 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:33.719+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:34.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:34 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:34.202+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:34.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:34 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:34.202+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:34.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:34 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:34.555+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:34.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:34 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:34.555+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:35.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:34 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:34.714+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:35.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:34 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:34.714+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:35.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:35 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:35.178+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:35.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:35 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:35.178+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:35.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:35 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:35.590+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:35.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:35 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:35.590+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:36.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:35 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:35.680+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:36.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:35 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:35.680+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:36.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:36 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:36.191+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:36.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:36 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:36.191+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:36.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:36 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:36.639+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:36.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:36 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:36.639+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:37.032 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:36 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:36.720+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.901326+0000 front 2024-04-20T21:45:17.901534+0000 (oldest deadline 2024-04-20T21:46:19.601148+0000) 2024-04-20T21:46:37.033 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:36 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9[49580]: 2024-04-20T21:46:36.720+0000 7f46e5c00700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.503103+0000 front 2024-04-20T21:45:25.503029+0000 (oldest deadline 2024-04-20T21:46:30.203244+0000) 2024-04-20T21:46:37.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:37 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:37.146+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:37.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:37 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:37.146+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:37.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:37 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:37.684+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:37.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:37 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:37.684+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:38.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:38.148+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:38.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:38 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:38.148+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:38.490 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:38 smithi007 podman[83231]: 2024-04-20 21:46:38.221370509 +0000 UTC m=+5.900280947 container died b842bfb6bd142a0571b65369e4f0a14b6ae144ee44de607e7ead0278d0984a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-20T21:46:38.782 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:38 smithi007 podman[83231]: 2024-04-20 21:46:38.612538847 +0000 UTC m=+6.291449305 container cleanup b842bfb6bd142a0571b65369e4f0a14b6ae144ee44de607e7ead0278d0984a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2) 2024-04-20T21:46:38.783 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:38 smithi007 bash[83231]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9 2024-04-20T21:46:38.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:38.663+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:38.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:38 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:38.663+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:39.282 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:38 smithi007 podman[83252]: 2024-04-20 21:46:38.81339468 +0000 UTC m=+0.577158622 container remove b842bfb6bd142a0571b65369e4f0a14b6ae144ee44de607e7ead0278d0984a3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:39.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:39 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:39.130+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:39.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:39 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:39.130+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:39.883 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:39 smithi007 podman[83347]: 2024-04-20 21:46:39.477548051 +0000 UTC m=+0.033963557 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:39.883 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:39 smithi007 podman[83347]: 2024-04-20 21:46:39.572754777 +0000 UTC m=+0.129170340 container create 1d480edc2be0528c3e75131ff0426df936d2a10bd6e367fb4cdbb0b84b263f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:46:39.883 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:39 smithi007 podman[83347]: 2024-04-20 21:46:39.884206352 +0000 UTC m=+0.440621878 container init 1d480edc2be0528c3e75131ff0426df936d2a10bd6e367fb4cdbb0b84b263f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD) 2024-04-20T21:46:39.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:39 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:39.636+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:39.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:39 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:39.636+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:40.283 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:39 smithi007 podman[83347]: 2024-04-20 21:46:39.896350889 +0000 UTC m=+0.452766449 container start 1d480edc2be0528c3e75131ff0426df936d2a10bd6e367fb4cdbb0b84b263f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:46:40.283 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:39 smithi007 podman[83347]: 2024-04-20 21:46:39.975297567 +0000 UTC m=+0.531713116 container attach 1d480edc2be0528c3e75131ff0426df936d2a10bd6e367fb4cdbb0b84b263f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:40.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:40 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:40.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:40.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:40 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:40.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:40.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:40 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:40.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:40.658 INFO:journalctl@ceph.osd.9.smithi007.stdout:Apr 20 21:46:40 smithi007 podman[83347]: 2024-04-20 21:46:40.372900899 +0000 UTC m=+0.929316436 container died 1d480edc2be0528c3e75131ff0426df936d2a10bd6e367fb4cdbb0b84b263f61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:40.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:40 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:40.681+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:40.945 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:40 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:40.681+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:40.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.9.service' 2024-04-20T21:46:41.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:41.101 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-20T21:46:41.101 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-20T21:46:41.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.10 2024-04-20T21:46:41.441 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 systemd[1]: Stopping Ceph osd.10 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:41.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:41 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:41.098+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:41.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:41 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:41.098+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:41.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:41 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:41.098+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:41.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.442+0000 7f8bf34a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:41.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.442+0000 7f8bf34a4700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-20T21:46:41.695 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.442+0000 7f8bf34a4700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:41.695 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.645+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:41.695 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.645+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:41.695 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:41 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:41.645+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:35.396117+0000 front 2024-04-20T21:45:35.395994+0000 (oldest deadline 2024-04-20T21:46:40.695912+0000) 2024-04-20T21:46:42.215 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:42 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:42.076+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:42.216 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:42 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:42.076+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:42.216 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:42 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:42.076+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:42.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:42 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:42.685+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:42.945 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:42 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:42.685+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:42.945 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:42 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:42.685+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:35.396117+0000 front 2024-04-20T21:45:35.395994+0000 (oldest deadline 2024-04-20T21:46:40.695912+0000) 2024-04-20T21:46:43.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:43 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:43.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:43.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:43 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:43.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:43.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:43 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:43.088+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:43.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:43 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:43.693+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:43.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:43 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:43.693+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:43.945 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:43 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:43.693+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:35.396117+0000 front 2024-04-20T21:45:35.395994+0000 (oldest deadline 2024-04-20T21:46:40.695912+0000) 2024-04-20T21:46:44.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:44 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:44.084+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:44.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:44 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:44.084+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:44.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:44 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:44.084+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:44.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:44 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:44.673+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:44.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:44 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:44.673+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:44.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:44 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:44.673+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:35.396117+0000 front 2024-04-20T21:45:35.395994+0000 (oldest deadline 2024-04-20T21:46:40.695912+0000) 2024-04-20T21:46:45.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:45 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:45.052+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:45.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:45 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:45.052+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:45.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:45 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:45.052+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:46.194 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:45 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:45.722+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:13.792361+0000 front 2024-04-20T21:45:13.792547+0000 (oldest deadline 2024-04-20T21:46:19.092313+0000) 2024-04-20T21:46:46.195 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:45 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:45.722+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:22.494374+0000 front 2024-04-20T21:45:22.494399+0000 (oldest deadline 2024-04-20T21:46:28.394392+0000) 2024-04-20T21:46:46.195 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:45 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10[46303]: 2024-04-20T21:46:45.722+0000 7f8beba90700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:35.396117+0000 front 2024-04-20T21:45:35.395994+0000 (oldest deadline 2024-04-20T21:46:40.695912+0000) 2024-04-20T21:46:46.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:46 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:46.075+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:46.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:46 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:46.075+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:46.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:46 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:46.075+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:47.291 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 podman[58170]: 2024-04-20 21:46:47.041588562 +0000 UTC m=+5.741292791 container died 81bf10d33ae4d690a26cff1c596df010a54ea8905070768418aa94838700c0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:47.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:47 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:47.087+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:47.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:47 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:47.087+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:47.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:47 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:47.087+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:47.566 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 podman[58170]: 2024-04-20 21:46:47.375546417 +0000 UTC m=+6.075250640 container cleanup 81bf10d33ae4d690a26cff1c596df010a54ea8905070768418aa94838700c0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:47.567 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 bash[58170]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10 2024-04-20T21:46:47.944 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 podman[58187]: 2024-04-20 21:46:47.568369813 +0000 UTC m=+0.526100288 container remove 81bf10d33ae4d690a26cff1c596df010a54ea8905070768418aa94838700c0da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:46:48.294 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 podman[58274]: 2024-04-20 21:46:47.877048219 +0000 UTC m=+0.018668153 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:48.295 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:47 smithi016 podman[58274]: 2024-04-20 21:46:47.985797269 +0000 UTC m=+0.127417202 container create 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T21:46:48.295 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:48 smithi016 podman[58274]: 2024-04-20 21:46:48.213338071 +0000 UTC m=+0.354958007 container init 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:48.295 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:48 smithi016 podman[58274]: 2024-04-20 21:46:48.217045151 +0000 UTC m=+0.358665094 container start 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:48.295 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:48 smithi016 podman[58274]: 2024-04-20 21:46:48.296570458 +0000 UTC m=+0.438190413 container attach 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-20T21:46:48.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:48 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:48.047+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:48.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:48 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:48.047+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:48.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:48 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:48.047+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:48.694 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:48 smithi016 podman[58274]: 2024-04-20 21:46:48.518319811 +0000 UTC m=+0.659939738 container died 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-20T21:46:49.122 INFO:journalctl@ceph.osd.10.smithi016.stdout:Apr 20 21:46:49 smithi016 podman[58300]: 2024-04-20 21:46:49.099563993 +0000 UTC m=+0.575559081 container remove 8b4522769fd57265ff6f65a324f2103a73bb2201ba996372142f546e97a3a45d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-20T21:46:49.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.10.service' 2024-04-20T21:46:49.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:49.172 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-20T21:46:49.172 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-20T21:46:49.173 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.11 2024-04-20T21:46:49.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.066+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:49.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.066+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:49.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.066+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:49.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 systemd[1]: Stopping Ceph osd.11 for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:49.965 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.470+0000 7fb911d79700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:49.965 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.470+0000 7fb911d79700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-20T21:46:49.966 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:49 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:49.470+0000 7fb911d79700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-20T21:46:50.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:50 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:50.095+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:50.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:50 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:50.095+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:50.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:50 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:50.095+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:51.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:51 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:51.068+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:51.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:51 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:51.068+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:51.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:51 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:51.068+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:52.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:52 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:52.079+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:52.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:52 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:52.079+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:52.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:52 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:52.079+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:53.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:53 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:53.050+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:53.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:53 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:53.050+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:53.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:53 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:53.050+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:54.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:54 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:54.073+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.7:6806 osd.0 since back 2024-04-20T21:45:17.097421+0000 front 2024-04-20T21:45:17.097323+0000 (oldest deadline 2024-04-20T21:46:21.197317+0000) 2024-04-20T21:46:54.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:54 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:54.073+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.16:6804 osd.1 since back 2024-04-20T21:45:25.898682+0000 front 2024-04-20T21:45:25.898788+0000 (oldest deadline 2024-04-20T21:46:31.198563+0000) 2024-04-20T21:46:54.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:54 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11[47760]: 2024-04-20T21:46:54.073+0000 7fb90e36d700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.186:6804 osd.2 since back 2024-04-20T21:45:33.499448+0000 front 2024-04-20T21:45:33.499424+0000 (oldest deadline 2024-04-20T21:46:39.399637+0000) 2024-04-20T21:46:55.489 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:55 smithi186 podman[60046]: 2024-04-20 21:46:55.223545988 +0000 UTC m=+5.860068758 container died c6005db1a0ba19b2a137686c1f779d6572a8f77be1d5a649b6b9edf72b86d1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-20T21:46:55.870 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:55 smithi186 podman[60046]: 2024-04-20 21:46:55.578692807 +0000 UTC m=+6.215215576 container cleanup c6005db1a0ba19b2a137686c1f779d6572a8f77be1d5a649b6b9edf72b86d1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-20T21:46:55.870 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:55 smithi186 bash[60046]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11 2024-04-20T21:46:56.184 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:55 smithi186 podman[60063]: 2024-04-20 21:46:55.871623049 +0000 UTC m=+0.646755365 container remove c6005db1a0ba19b2a137686c1f779d6572a8f77be1d5a649b6b9edf72b86d1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2) 2024-04-20T21:46:56.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.089410547 +0000 UTC m=+0.018076718 image pull 76bc6b91b726a03b8b736eeeb35fc1ffd1469e23c7d60f5b444a217528d8906d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1 2024-04-20T21:46:56.465 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.189020641 +0000 UTC m=+0.117686808 container create ed9a74bab7a62d99ebf46c9f5c0fd18845033b83332bb955fc17c58ad990faa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-20T21:46:56.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.399962016 +0000 UTC m=+0.328628174 container init ed9a74bab7a62d99ebf46c9f5c0fd18845033b83332bb955fc17c58ad990faa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:46:56.466 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.402950703 +0000 UTC m=+0.331616858 container start ed9a74bab7a62d99ebf46c9f5c0fd18845033b83332bb955fc17c58ad990faa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-20T21:46:56.965 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.481805402 +0000 UTC m=+0.410471568 container attach ed9a74bab7a62d99ebf46c9f5c0fd18845033b83332bb955fc17c58ad990faa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-20T21:46:56.966 INFO:journalctl@ceph.osd.11.smithi186.stdout:Apr 20 21:46:56 smithi186 podman[60154]: 2024-04-20 21:46:56.704221095 +0000 UTC m=+0.632887244 container died ed9a74bab7a62d99ebf46c9f5c0fd18845033b83332bb955fc17c58ad990faa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-20T21:46:57.266 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@osd.11.service' 2024-04-20T21:46:57.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:46:57.304 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-20T21:46:57.305 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-20T21:46:57.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.a 2024-04-20T21:46:57.782 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:57 smithi007 systemd[1]: Stopping Ceph mds.a for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:46:58.282 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:57 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a[56241]: 2024-04-20T21:46:57.895+0000 7f1868800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:46:58.283 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:57 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a[56241]: 2024-04-20T21:46:57.895+0000 7f1868800700 -1 mds.a *** got signal Terminated *** 2024-04-20T21:46:59.782 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:59 smithi007 podman[83453]: 2024-04-20 21:46:59.526691788 +0000 UTC m=+1.754086316 container died f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-20T21:47:00.121 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:59 smithi007 podman[83453]: 2024-04-20 21:46:59.92988145 +0000 UTC m=+2.157275974 container cleanup f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-20T21:47:00.122 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:46:59 smithi007 bash[83453]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a 2024-04-20T21:47:00.532 INFO:journalctl@ceph.mds.a.smithi007.stdout:Apr 20 21:47:00 smithi007 podman[83468]: 2024-04-20 21:47:00.122472799 +0000 UTC m=+0.576554919 container remove f1869a100f95b907de7d0fed8ea525aad7c3aafd620022cdd1a13136296638ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-20T21:47:00.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.a.service' 2024-04-20T21:47:00.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:00.772 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-20T21:47:00.772 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-20T21:47:00.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.d 2024-04-20T21:47:01.282 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:00 smithi007 systemd[1]: Stopping Ceph mds.d for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:01.782 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:01 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d[55222]: 2024-04-20T21:47:01.373+0000 7feab4200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:01.783 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:01 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d[55222]: 2024-04-20T21:47:01.373+0000 7feab4200700 -1 mds.d *** got signal Terminated *** 2024-04-20T21:47:04.783 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:04 smithi007 podman[83571]: 2024-04-20 21:47:04.513190432 +0000 UTC m=+3.293071141 container died d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-20T21:47:05.091 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:04 smithi007 podman[83571]: 2024-04-20 21:47:04.899756018 +0000 UTC m=+3.679636731 container cleanup d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:05.092 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:04 smithi007 bash[83571]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d 2024-04-20T21:47:05.532 INFO:journalctl@ceph.mds.d.smithi007.stdout:Apr 20 21:47:05 smithi007 podman[83585]: 2024-04-20 21:47:05.092332357 +0000 UTC m=+0.558840195 container remove d1255bd25d6e2d51280aafa329e95699f53f10e13d26f7b1e85c9b6279fc8963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-20T21:47:05.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.d.service' 2024-04-20T21:47:05.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:05.722 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-20T21:47:05.722 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-20T21:47:05.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.g 2024-04-20T21:47:06.282 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:05 smithi007 systemd[1]: Stopping Ceph mds.g for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:06.676 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g[55873]: 2024-04-20T21:47:06.335+0000 7fc915800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:06.676 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:06 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g[55873]: 2024-04-20T21:47:06.335+0000 7fc915800700 -1 mds.g *** got signal Terminated *** 2024-04-20T21:47:07.640 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:07 smithi007 podman[83692]: 2024-04-20 21:47:07.329294773 +0000 UTC m=+1.128073928 container died 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-20T21:47:07.960 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:07 smithi007 podman[83692]: 2024-04-20 21:47:07.778387163 +0000 UTC m=+1.577166325 container cleanup 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-20T21:47:07.960 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:07 smithi007 bash[83692]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g 2024-04-20T21:47:08.282 INFO:journalctl@ceph.mds.g.smithi007.stdout:Apr 20 21:47:08 smithi007 podman[83715]: 2024-04-20 21:47:08.063133231 +0000 UTC m=+0.720000338 container remove 96fe345f286f0e71df43ccd6fcee5e368099f30dc3509c72af0092ebbdf2a7e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-20T21:47:08.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.g.service' 2024-04-20T21:47:08.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:08.764 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-20T21:47:08.765 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-20T21:47:08.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.j 2024-04-20T21:47:09.033 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:08 smithi007 systemd[1]: Stopping Ceph mds.j for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:09.783 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j[54877]: 2024-04-20T21:47:09.339+0000 7fd0bac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:09.783 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:09 smithi007 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j[54877]: 2024-04-20T21:47:09.339+0000 7fd0bac00700 -1 mds.j *** got signal Terminated *** 2024-04-20T21:47:12.744 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:12 smithi007 podman[83824]: 2024-04-20 21:47:12.467663053 +0000 UTC m=+3.261124704 container died 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-20T21:47:12.999 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:12 smithi007 podman[83824]: 2024-04-20 21:47:12.9740281 +0000 UTC m=+3.767489777 container cleanup 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:47:12.999 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:12 smithi007 bash[83824]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j 2024-04-20T21:47:13.282 INFO:journalctl@ceph.mds.j.smithi007.stdout:Apr 20 21:47:13 smithi007 podman[83838]: 2024-04-20 21:47:13.19182522 +0000 UTC m=+0.700728793 container remove 660811f038160d76e744db1c543010fd11d0808508b82139301420305c45d5ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:13.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.j.service' 2024-04-20T21:47:13.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:13.798 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-20T21:47:13.798 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-20T21:47:13.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.b 2024-04-20T21:47:13.944 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:13 smithi016 systemd[1]: Stopping Ceph mds.b for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:14.444 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: 2024-04-20T21:47:14.108+0000 7f9890397700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:14.444 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:14 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b[51701]: 2024-04-20T21:47:14.108+0000 7f9890397700 -1 mds.b *** got signal Terminated *** 2024-04-20T21:47:18.341 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:18 smithi016 podman[58393]: 2024-04-20 21:47:18.071429473 +0000 UTC m=+4.072963854 container died 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T21:47:18.694 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:18 smithi016 podman[58393]: 2024-04-20 21:47:18.575825445 +0000 UTC m=+4.577359835 container cleanup 92d747a36083ba8934177d763fea372f6fc882e82222293d03298d18dcbb5c4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T21:47:18.694 INFO:journalctl@ceph.mds.b.smithi016.stdout:Apr 20 21:47:18 smithi016 bash[58393]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-b 2024-04-20T21:47:19.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.b.service' 2024-04-20T21:47:19.042 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:19.042 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-20T21:47:19.042 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-20T21:47:19.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.e 2024-04-20T21:47:19.444 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:19 smithi016 systemd[1]: Stopping Ceph mds.e for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:19.444 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:19 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: 2024-04-20T21:47:19.353+0000 7f233f48e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:19.444 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:19 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e[51072]: 2024-04-20T21:47:19.353+0000 7f233f48e700 -1 mds.e *** got signal Terminated *** 2024-04-20T21:47:22.328 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:22 smithi016 podman[58515]: 2024-04-20 21:47:22.065100401 +0000 UTC m=+2.801669400 container died f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-20T21:47:22.694 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:22 smithi016 podman[58515]: 2024-04-20 21:47:22.429187519 +0000 UTC m=+3.165756528 container cleanup f54d6258245547ae1f1427d59ddaf26162c4842f2a6de1ffd51d5f00859dd6a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-20T21:47:22.694 INFO:journalctl@ceph.mds.e.smithi016.stdout:Apr 20 21:47:22 smithi016 bash[58515]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-e 2024-04-20T21:47:22.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.e.service' 2024-04-20T21:47:22.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:22.871 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-20T21:47:22.871 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-20T21:47:22.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.h 2024-04-20T21:47:23.194 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:22 smithi016 systemd[1]: Stopping Ceph mds.h for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:23.694 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:23 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: 2024-04-20T21:47:23.214+0000 7f922c1ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:23.694 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:23 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h[51381]: 2024-04-20T21:47:23.214+0000 7f922c1ba700 -1 mds.h *** got signal Terminated *** 2024-04-20T21:47:26.373 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:26 smithi016 podman[58638]: 2024-04-20 21:47:26.080017201 +0000 UTC m=+2.977483877 container died ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-20T21:47:26.694 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:26 smithi016 podman[58638]: 2024-04-20 21:47:26.474362162 +0000 UTC m=+3.371828838 container cleanup ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:26.694 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:26 smithi016 bash[58638]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h 2024-04-20T21:47:27.041 INFO:journalctl@ceph.mds.h.smithi016.stdout:Apr 20 21:47:26 smithi016 podman[58650]: 2024-04-20 21:47:26.783988416 +0000 UTC m=+0.706122030 container remove ed072abb3a762ba988803f9c6529e90d294ba995c7941546aebbd1dbb40250fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True) 2024-04-20T21:47:27.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.h.service' 2024-04-20T21:47:27.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:27.093 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-20T21:47:27.094 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-20T21:47:27.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.k 2024-04-20T21:47:27.417 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:27 smithi016 systemd[1]: Stopping Ceph mds.k for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:27.694 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:27 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k[50754]: 2024-04-20T21:47:27.418+0000 7fc3d26a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:27.694 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:27 smithi016 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k[50754]: 2024-04-20T21:47:27.418+0000 7fc3d26a4700 -1 mds.k *** got signal Terminated *** 2024-04-20T21:47:28.867 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:28 smithi016 podman[58761]: 2024-04-20 21:47:28.548588246 +0000 UTC m=+1.231171504 container died 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-20T21:47:28.868 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:28 smithi016 podman[58761]: 2024-04-20 21:47:28.869727992 +0000 UTC m=+1.552311246 container cleanup 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-20T21:47:29.194 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:28 smithi016 bash[58761]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k 2024-04-20T21:47:29.194 INFO:journalctl@ceph.mds.k.smithi016.stdout:Apr 20 21:47:29 smithi016 podman[58775]: 2024-04-20 21:47:29.078961042 +0000 UTC m=+0.527915055 container remove 2b7dbfab4e260d6a5c9a3d85dabc2b6427e66cbe0a95948422481d8b677bb8d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.schema-version=1.0) 2024-04-20T21:47:29.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.k.service' 2024-04-20T21:47:29.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:29.329 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-20T21:47:29.329 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-20T21:47:29.329 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.c 2024-04-20T21:47:29.627 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:29 smithi186 systemd[1]: Stopping Ceph mds.c for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:29.965 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:29 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: 2024-04-20T21:47:29.628+0000 7fec34225700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:29.965 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:29 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c[53095]: 2024-04-20T21:47:29.628+0000 7fec34225700 -1 mds.c *** got signal Terminated *** 2024-04-20T21:47:33.894 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:33 smithi186 podman[60261]: 2024-04-20 21:47:33.640443899 +0000 UTC m=+4.119638650 container died 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True) 2024-04-20T21:47:34.215 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:33 smithi186 podman[60261]: 2024-04-20 21:47:33.995378121 +0000 UTC m=+4.474572875 container cleanup 4a0ce48fda1b9be0bc908acce0c981dd49451d77ce85d634b30895794af9b404 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5) 2024-04-20T21:47:34.215 INFO:journalctl@ceph.mds.c.smithi186.stdout:Apr 20 21:47:33 smithi186 bash[60261]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-c 2024-04-20T21:47:34.428 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.c.service' 2024-04-20T21:47:34.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:34.463 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-20T21:47:34.463 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-20T21:47:34.463 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.f 2024-04-20T21:47:34.790 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:34 smithi186 systemd[1]: Stopping Ceph mds.f for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:35.215 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:34 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: 2024-04-20T21:47:34.791+0000 7fb39772a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:35.215 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:34 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f[52784]: 2024-04-20T21:47:34.791+0000 7fb39772a700 -1 mds.f *** got signal Terminated *** 2024-04-20T21:47:38.359 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:38 smithi186 podman[60394]: 2024-04-20 21:47:38.054503578 +0000 UTC m=+3.366764991 container died fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-20T21:47:38.657 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:38 smithi186 podman[60394]: 2024-04-20 21:47:38.449814387 +0000 UTC m=+3.762075798 container cleanup fc2064cada42e6c84d3c84b2d27ce3bf32c9ba233383ead5be0ffc3cd0c68a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, maintainer=Guillaume Abrioux ) 2024-04-20T21:47:38.657 INFO:journalctl@ceph.mds.f.smithi186.stdout:Apr 20 21:47:38 smithi186 bash[60394]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-f 2024-04-20T21:47:38.805 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.f.service' 2024-04-20T21:47:38.838 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:38.839 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-20T21:47:38.839 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-20T21:47:38.839 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.i 2024-04-20T21:47:39.215 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:38 smithi186 systemd[1]: Stopping Ceph mds.i for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:39.215 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:39 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: 2024-04-20T21:47:39.152+0000 7efe95cf5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:39.216 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:39 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i[53415]: 2024-04-20T21:47:39.152+0000 7efe95cf5700 -1 mds.i *** got signal Terminated *** 2024-04-20T21:47:42.215 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:42 smithi186 podman[60514]: 2024-04-20 21:47:42.06988217 +0000 UTC m=+3.010758532 container died 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-20T21:47:42.688 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:42 smithi186 podman[60514]: 2024-04-20 21:47:42.447649745 +0000 UTC m=+3.388526113 container cleanup 08bc8d6ae2a775cc71b6bd2b6d2b9508335f46d9cef20a825956012dbac703a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-20T21:47:42.688 INFO:journalctl@ceph.mds.i.smithi186.stdout:Apr 20 21:47:42 smithi186 bash[60514]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-i 2024-04-20T21:47:42.828 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.i.service' 2024-04-20T21:47:42.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:42.860 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-20T21:47:42.861 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-20T21:47:42.861 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.l 2024-04-20T21:47:43.200 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:42 smithi186 systemd[1]: Stopping Ceph mds.l for 8e9da876-ff5b-11ee-bc93-c7b262605968... 2024-04-20T21:47:43.465 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:43 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l[52479]: 2024-04-20T21:47:43.200+0000 7fd2e3693700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-20T21:47:43.465 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:43 smithi186 ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l[52479]: 2024-04-20T21:47:43.200+0000 7fd2e3693700 -1 mds.l *** got signal Terminated *** 2024-04-20T21:47:45.858 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:45 smithi186 podman[60633]: 2024-04-20 21:47:45.609563234 +0000 UTC m=+2.527168880 container died caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-20T21:47:46.133 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:45 smithi186 podman[60633]: 2024-04-20 21:47:45.959274782 +0000 UTC m=+2.876880433 container cleanup caa98babf1cd71591c8bbc5a9730f5e82f963198ee9f80ff0190f0729008ee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:556b2d4db8c1fdd86d8938d44019b738fe91b025623acee0a51cae93523136b1, name=ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-a8c84f5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-20T21:47:46.133 INFO:journalctl@ceph.mds.l.smithi186.stdout:Apr 20 21:47:45 smithi186 bash[60633]: ceph-8e9da876-ff5b-11ee-bc93-c7b262605968-mds-l 2024-04-20T21:47:46.342 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8e9da876-ff5b-11ee-bc93-c7b262605968@mds.l.service' 2024-04-20T21:47:46.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-20T21:47:46.373 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-20T21:47:46.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:47:51.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:47:51.972 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:47:54.508 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-20T21:47:54.701 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:47:57.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:47:57.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:47:57.307 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-20T21:47:57.332 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-20T21:47:57.335 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi007/crash 2024-04-20T21:47:57.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:47:57.439 INFO:teuthology.orchestra.run.smithi007.stderr:tar: /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:47:57.440 INFO:teuthology.orchestra.run.smithi007.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:47:57.444 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi016/crash 2024-04-20T21:47:57.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:47:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:47:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:47:57.494 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi186/crash 2024-04-20T21:47:57.495 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash -- . 2024-04-20T21:47:57.533 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-20T21:47:57.533 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2024-04-20T21:47:57.534 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-20T21:47:57.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-20T21:47:57.704 INFO:teuthology.orchestra.run.smithi007.stdout:2024-04-20T21:37:42.311614+0000 mon.a (mon.0) 1165 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-20T21:47:57.706 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-20T21:47:57.706 INFO:tasks.cephadm:Compressing logs... 2024-04-20T21:47:57.706 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:47:57.711 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:47:57.714 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:47:57.740 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:47:57.741 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2024-04-20T21:47:57.741 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-20T21:47:57.741 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-20T21:47:57.741 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-20T21:47:57.742 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-20T21:47:57.743 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:47:57.743 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-20T21:47:57.743 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:47:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:47:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-04-20T21:47:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:47:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-20T21:47:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-20T21:47:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-20T21:47:57.746 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-20T21:47:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:47:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-20T21:47:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-20T21:47:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:47:57.788 INFO:teuthology.orchestra.run.smithi007.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-20T21:47:57.788 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log 2024-04-20T21:47:57.789 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-20T21:47:57.790 INFO:teuthology.orchestra.run.smithi186.stderr: 83.4% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:47:57.791 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-20T21:47:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 94.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-20T21:47:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-20T21:47:57.793 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log: 87.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:47:57.793 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-20T21:47:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-20T21:47:57.796 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-20T21:47:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-20T21:47:57.799 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-20T21:47:57.799 INFO:teuthology.orchestra.run.smithi007.stderr: 94.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-20T21:47:57.799 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-20T21:47:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-20T21:47:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-20T21:47:57.802 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-20T21:47:57.803 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-20T21:47:57.804 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-20T21:47:57.805 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-20T21:47:57.807 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-20T21:47:57.809 INFO:teuthology.orchestra.run.smithi007.stderr: 94.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-20T21:47:57.842 INFO:teuthology.orchestra.run.smithi186.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:47:57.855 INFO:teuthology.orchestra.run.smithi016.stderr: 92.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-20T21:47:57.888 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:47:57.897 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:47:57.958 INFO:teuthology.orchestra.run.smithi007.stderr: 90.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:47:57.964 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:47:57.992 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:47:57.997 INFO:teuthology.orchestra.run.smithi186.stderr: 92.7% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-20T21:47:58.026 INFO:teuthology.orchestra.run.smithi186.stderr: 87.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:47:58.043 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-20T21:47:58.059 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-20T21:47:58.063 INFO:teuthology.orchestra.run.smithi016.stderr: 83.5% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:47:58.072 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-20T21:47:58.093 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-20T21:47:58.110 INFO:teuthology.orchestra.run.smithi186.stderr: 90.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-20T21:47:58.113 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-20T21:47:58.134 INFO:teuthology.orchestra.run.smithi186.stderr: 95.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-20T21:47:58.214 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-20T21:47:58.345 INFO:teuthology.orchestra.run.smithi007.stderr: 83.9% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-20T21:47:58.533 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-20T21:47:59.262 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-20T21:48:01.890 INFO:teuthology.orchestra.run.smithi186.stderr: 91.8% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-20T21:48:02.201 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-20T21:48:02.414 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-20T21:48:02.780 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-20T21:49:02.644 INFO:teuthology.orchestra.run.smithi016.stderr: 92.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-20T21:49:06.192 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-20T21:49:12.758 INFO:teuthology.orchestra.run.smithi186.stderr: 92.3% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-20T21:49:24.883 INFO:teuthology.orchestra.run.smithi016.stderr: 92.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-20T21:49:59.524 INFO:teuthology.orchestra.run.smithi186.stderr: 92.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-20T21:50:07.760 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-20T21:50:10.005 INFO:teuthology.orchestra.run.smithi186.stderr: 92.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-20T21:50:10.007 INFO:teuthology.orchestra.run.smithi186.stderr: 2024-04-20T21:50:10.007 INFO:teuthology.orchestra.run.smithi186.stderr:real 2m12.279s 2024-04-20T21:50:10.007 INFO:teuthology.orchestra.run.smithi186.stderr:user 4m10.128s 2024-04-20T21:50:10.007 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m8.509s 2024-04-20T21:50:24.709 INFO:teuthology.orchestra.run.smithi016.stderr: 92.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-20T21:50:24.711 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-04-20T21:50:24.711 INFO:teuthology.orchestra.run.smithi016.stderr:real 2m26.982s 2024-04-20T21:50:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:user 4m30.871s 2024-04-20T21:50:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m8.443s 2024-04-20T21:50:25.320 INFO:teuthology.orchestra.run.smithi007.stderr: 92.4% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-20T21:50:46.169 INFO:teuthology.orchestra.run.smithi007.stderr: 92.2% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-20T21:51:16.733 INFO:teuthology.orchestra.run.smithi007.stderr: 92.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-20T21:51:18.527 INFO:teuthology.orchestra.run.smithi007.stderr: 92.1% -- replaced with /var/log/ceph/8e9da876-ff5b-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-20T21:51:18.532 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-20T21:51:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:real 3m20.778s 2024-04-20T21:51:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:user 6m21.225s 2024-04-20T21:51:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m40.218s 2024-04-20T21:51:18.534 INFO:tasks.cephadm:Archiving logs... 2024-04-20T21:51:18.535 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi007/log 2024-04-20T21:51:18.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:51:58.545 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi016/log 2024-04-20T21:51:58.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:52:42.691 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi186/log 2024-04-20T21:52:42.692 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:53:22.611 INFO:tasks.cephadm:Removing cluster... 2024-04-20T21:53:22.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:53:22.948 INFO:teuthology.orchestra.run.smithi007.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:53:24.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:53:24.505 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:53:25.069 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8e9da876-ff5b-11ee-bc93-c7b262605968 --force 2024-04-20T21:53:25.262 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 8e9da876-ff5b-11ee-bc93-c7b262605968 2024-04-20T21:53:25.830 INFO:tasks.cephadm:Removing cephadm ... 2024-04-20T21:53:25.831 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:53:25.892 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:53:25.911 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-20T21:53:25.927 INFO:tasks.cephadm:Teardown complete 2024-04-20T21:53:25.928 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T21:53:25.939 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T21:53:25.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:53:25.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:53:25.954 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:53:26.117 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T21:53:26.129 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T21:53:26.129 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:53:26.137 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:53:26.140 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:53:26.154 INFO:teuthology.orchestra.run.smithi186.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:53:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:53:26.173 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:53:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-04-20T21:53:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 -828us[ -828us] +/- 85ms 2024-04-20T21:53:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 8 377 179 -553us[ -562us] +/- 64ms 2024-04-20T21:53:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +1505us[+1505us] +/- 74ms 2024-04-20T21:53:26.174 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 -941us[ -959us] +/- 85ms 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -632us[ -649us] +/- 66ms 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 58 +1411us[+1394us] +/- 74ms 2024-04-20T21:53:26.183 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:53:26.184 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:53:26.194 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:53:26.194 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-04-20T21:53:26.194 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 -873us[ -873us] +/- 85ms 2024-04-20T21:53:26.195 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -632us[ -621us] +/- 66ms 2024-04-20T21:53:26.195 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +1458us[+1458us] +/- 74ms 2024-04-20T21:53:26.195 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:53:26.196 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T21:53:26.206 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T21:53:26.207 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T21:53:26.217 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:53:26.448 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:53:26.522 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:53:26.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:53:26.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:26.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:53:26.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:26.713 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:53:26.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:26.740 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T21:53:26.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T21:53:26.762 INFO:teuthology.task.internal:Duration was 2476.269695 seconds 2024-04-20T21:53:26.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T21:53:26.773 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T21:53:26.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:53:26.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:53:26.783 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:53:26.835 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:53:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:53:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:53:27.266 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T21:53:27.267 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-04-20T21:53:27.267 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:53:27.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-04-20T21:53:27.347 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:53:27.380 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2024-04-20T21:53:27.380 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:53:27.406 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T21:53:27.406 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:53:27.412 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:53:27.423 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:53:27.499 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T21:53:27.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:53:27.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:53:27.507 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:53:27.974 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T21:53:27.985 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T21:53:27.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:53:28.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:53:28.131 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:53:28.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T21:53:28.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.234 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.258 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-04-20T21:53:28.262 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2024-04-20T21:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-04-20T21:53:28.384 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:28.436 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:28.452 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:53:28.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:53:28.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T21:53:28.802 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T21:53:29.094 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi007 2024-04-20T21:53:29.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:53:29.205 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi016 2024-04-20T21:53:29.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:53:29.243 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396/remote/smithi186 2024-04-20T21:53:29.243 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:53:29.277 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T21:53:29.278 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:53:29.285 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:53:29.288 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:53:29.339 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T21:53:29.349 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T21:53:29.350 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T21:53:29.360 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T21:53:29.360 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:53:29.366 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:53:29.368 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:53:29.384 INFO:teuthology.orchestra.run.smithi016.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest 2024-04-20T21:53:29.384 INFO:teuthology.orchestra.run.smithi186.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest 2024-04-20T21:53:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: 393514 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:53 /home/ubuntu/cephtest 2024-04-20T21:53:29.422 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T21:53:29.433 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T21:53:29.456 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T21:53:29.482 DEBUG:teuthology.nuke:shortname: smithi007 2024-04-20T21:53:29.482 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:53:29.507 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.315764', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVLjm1H8OunaUtqlqULlwQaqtVnsxCBU0JXx1+QLcvtdZeRJdMPRlcz4Z/kKFcZKtKqyn+6db0D6wTRXtVVSUM='} 2024-04-20T21:53:29.511 DEBUG:teuthology.nuke:shortname: smithi016 2024-04-20T21:53:29.511 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:53:29.533 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.317350', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIrt8hvUjTFHy0+87Ds8QyWCaE2LCQtXTyt/sewtNUTFCmziszohH3r/QhHUe5wvmRzLkl7CCdEkyyPD1hnLhk0='} 2024-04-20T21:53:29.538 DEBUG:teuthology.nuke:shortname: smithi186 2024-04-20T21:53:29.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:53:29.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665396', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:40.318624', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIHplz7hWBXe4uK2eMcvNKEhfAiWfU5OH1plxMRpRIzKbzZ4iA+X9Fr86vGyaZ0Q11Stw89x+Xqw/OACmqEnwHI='} 2024-04-20T21:53:29.571 INFO:teuthology.orchestra.console.smithi007:Power off 2024-04-20T21:53:29.572 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:53:29.587 INFO:teuthology.orchestra.console.smithi016:Power off 2024-04-20T21:53:29.587 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:53:29.596 DEBUG:teuthology.orchestra.console.smithi007:power off output: Chassis Power Control: Down/Off 2024-04-20T21:53:29.596 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:29.618 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-04-20T21:53:29.619 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:29.626 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-04-20T21:53:29.634 INFO:teuthology.orchestra.console.smithi186:Power off 2024-04-20T21:53:29.635 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:53:29.644 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-04-20T21:53:29.657 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-04-20T21:53:29.657 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:29.683 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-20T21:53:33.631 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:33.645 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:33.685 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:33.741 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is on 2024-04-20T21:53:33.755 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-04-20T21:53:33.796 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-20T21:53:37.746 DEBUG:teuthology.orchestra.console.smithi007:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:37.756 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:37.796 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:37.857 DEBUG:teuthology.orchestra.console.smithi007:check power output: Chassis Power is off 2024-04-20T21:53:37.867 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-04-20T21:53:37.907 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-04-20T21:53:37.958 INFO:teuthology.orchestra.console.smithi007:Power off completed 2024-04-20T21:53:37.968 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-04-20T21:53:41.911 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:53:42.021 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-04-20T21:53:42.122 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-04-20T21:53:42.228 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} duration: 2476.2696948051453 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi007 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a8c84f5997950fd7d008204839253291d52e6eda TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=32523486170d42bba4ac50c0983dab44 status: fail success: false 2024-04-20T21:53:42.229 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:53:42.323 INFO:teuthology.run:FAIL